C# Client Library
A C# Client Library for the AnalyzeRe REST API
Loading...
Searching...
No Matches
AnalyzeRe.LargeDataUpload.AsyncParameters Class Reference

Parameters to be used in a Large Upload operation. More...

Public Types

enum  BinaryYELTUploadOptions { AutomaticBinaryConversionAndCompression , AutomaticCompressionOnly , None }
 Options for controlling whether to automatically convert a binary YELTLossSet's data to the binary format during upload. More...
 

Public Member Functions

 AsyncParameters ()
 Construct a new set of Default Large Data Async Upload Parameters.
 
 AsyncParameters (int? min_chunk_size=null, int? max_chunk_size=null, int? max_retries_per_chunk=null, int? chunk_timeout=null, PollingOptions commit_polling_options=null, HandleUploadInSessionStrategy? handle_existing_upload_strategy=null, bool? enable_compression=null, Action< StatusResponse > onSuccess=null, Action< Exception > onError=null, Action< StatusResponse > onProgressUpdate=null)
 Construct a new set of Large Data Async Upload Parameters.
 
 AsyncParameters (Parameters source, Action< StatusResponse > onSuccess=null, Action< Exception > onError=null, Action< StatusResponse > onProgressUpdate=null)
 Construct a new set of Large Data Async Upload Parameters from a set of Large Data Upload Parameters.
 

Public Attributes

Action< Exception > OnError
 Action to invoke upon failure of the large upload. If this is supplied, it will be called when an error occurs rather than the invoked method throwing an exception.
 
Action< StatusResponseOnProgressUpdate
 Action to invoke every time a chunk is uploaded. Callback is supplied with a double indicating progress towards completion (from 0 to 1).
 
Action< StatusResponseOnSuccess
 Action to invoke upon successful completion of the large upload.
 

Properties

BinaryYELTUploadOptions binary_yelt_options [get, set]
 Options for controlling whether to automatically convert a binary YELTLossSet's data to the binary format during upload. By default, the assumption is that csv data was provided and all conversion should be done on your behalf. Set to None if the file you're uploading is already in binary format and gzipped.
 
int chunk_timeout = 1000 * 60 [get, set]
 The timeout (in milliseconds) for a single chunk upload to complete. Default: 60,000 ms (1 minute).
 
PollingOptions commit_polling_options = DefaultCommitPollingOptions [get, set]
 Determines how the system polls the status of the data file while waiting for the data to be processed. By default, these options use exponential back-off and will timeout if the system hasn't processed the file in 3,600,000 ms (1 hour).
 
static new AsyncParameters Default [get]
 Get a new set of Default Async Large Data Upload Parameters.
 
static PollingOptions DefaultCommitPollingOptions [get]
 The default PollingOptions used when none are specified in a AnalyzeRe.LargeDataUpload.Parameters instance. By default, limits the maximum polling time to 1 hour. All other parameters use the default values for PollingOptions.
 
bool enable_compression [get, set]
 Whether to compress (gzip) the data during upload. (Default: False) Set to true if the files you're uploading are quite large and/or you wish to decrease network usage. In practice, if your upload speed is in excess of 5 MiBps, the overhead of compression and decompression on the server outweighs the reduction in uploaded data.
 
HandleUploadInSessionStrategy handle_existing_upload_strategy [get, set]
 The HandleUploadInSessionStrategy to employ of an existing upload session is already in progress. Default: RaiseError.
 
int max_chunk_size = 16 * 1024 * 1024 [get, set]
 The maximum size of a single uploaded chunk in bytes. Default: 2^24 bytes (16 Megabytes).
 
int max_retries_per_chunk = 3 [get, set]
 The maximum number of retries for a failed chunk upload before an error is thrown. Default: 3 retries.
 
int min_chunk_size = 4 * 1024 * 1024 [get, set]
 The minimum size of a single uploaded chunk in bytes. Set this larger if there is enough latency between the server that smaller chunk requests decrease your throughput. Set this smaller if your upload stream is expensive to produce and you want to reduce the amount of time before the first chunk gets sent over the network layer. Default: 2^22 bytes (4 Megabytes).
 

Detailed Description

Parameters to be used in a Large Upload operation.

Definition at line 6 of file AsyncParameters.cs.

Inheritance diagram for AnalyzeRe.LargeDataUpload.AsyncParameters:
AnalyzeRe.LargeDataUpload.Parameters

Member Enumeration Documentation

◆ BinaryYELTUploadOptions

Options for controlling whether to automatically convert a binary YELTLossSet's data to the binary format during upload.

Enumerator
AutomaticBinaryConversionAndCompression 

CSV YELT data was provided and should be converted automatically.

AutomaticCompressionOnly 

Binary YELT data was provided, but Gzip compression should be done automatically.

None 

Gzipped Binary YELT data was provided and the data should not be modified further.

Definition at line 66 of file Parameters.cs.

Constructor & Destructor Documentation

◆ AsyncParameters() [1/3]

AnalyzeRe.LargeDataUpload.AsyncParameters.AsyncParameters ( )
inline

Construct a new set of Default Large Data Async Upload Parameters.

Definition at line 29 of file AsyncParameters.cs.

◆ AsyncParameters() [2/3]

AnalyzeRe.LargeDataUpload.AsyncParameters.AsyncParameters ( int?  min_chunk_size = null,
int?  max_chunk_size = null,
int?  max_retries_per_chunk = null,
int?  chunk_timeout = null,
PollingOptions  commit_polling_options = null,
HandleUploadInSessionStrategy handle_existing_upload_strategy = null,
bool?  enable_compression = null,
Action< StatusResponse onSuccess = null,
Action< Exception >  onError = null,
Action< StatusResponse onProgressUpdate = null 
)
inline

Construct a new set of Large Data Async Upload Parameters.

Parameters
min_chunk_sizeThe minimum size of a single uploaded chunk in bytes. See Parameters.min_chunk_size for details and defaults.
max_chunk_sizeThe maximum size of a single uploaded chunk in bytes. See Parameters.max_chunk_size for details and defaults.
max_retries_per_chunkThe maximum number of retries for a failed chunk upload before an error is thrown. See Parameters.max_retries_per_chunk for details and defaults.
chunk_timeoutThe timeout (in milliseconds) for a single chunk upload to complete. See Parameters.chunk_timeout for details and defaults.
commit_polling_optionsDetermines how the system polls the status of the data file while waiting for the data to be processed. See Parameters.commit_polling_options for details and defaults.
handle_existing_upload_strategyThe HandleUploadInSessionStrategy to employ of an existing upload session is already in progress. See Parameters.handle_existing_upload_strategy for details and defaults.
enable_compressionWhether to compress (gzip) the data during upload. See Parameters.enable_compression for details and defaults.
onSuccessAn optional action to invoke upon successful completion of the large upload.
onErrorAn optional action to invoke upon failure of the large upload. If this is supplied, it will be called when an error occurs rather than the invoked method throwing an exception.
onProgressUpdateAn optional action to invoke every time a chunk is uploaded. Callback is supplied with a double indicating progress towards completion.

Definition at line 57 of file AsyncParameters.cs.

◆ AsyncParameters() [3/3]

AnalyzeRe.LargeDataUpload.AsyncParameters.AsyncParameters ( Parameters  source,
Action< StatusResponse onSuccess = null,
Action< Exception >  onError = null,
Action< StatusResponse onProgressUpdate = null 
)
inline

Construct a new set of Large Data Async Upload Parameters from a set of Large Data Upload Parameters.

Parameters
sourceThe Large Data Upload Parameters to get the non-Async parameters from.
onSuccessAn optional action to invoke upon successful completion of the large upload. (If the source is an AyncParameters instance and onSuccess is not overridden here, it will be copied from the source.)
onErrorAn optional action to invoke upon failure of the large upload. If this is supplied, it will be called when an error occurs rather than the invoked method throwing an exception. (If the source is an AyncParameters instance and onError is not overridden here, it will be copied from the source.)
onProgressUpdateAn optional action to invoke every time a chunk is uploaded. Callback is supplied with a double indicating progress towards completion. (If the source is an AyncParameters instance and onProgressUpdate is not overridden here, it will be copied from the source.)

Definition at line 91 of file AsyncParameters.cs.

Member Data Documentation

◆ OnError

Action<Exception> AnalyzeRe.LargeDataUpload.AsyncParameters.OnError

Action to invoke upon failure of the large upload. If this is supplied, it will be called when an error occurs rather than the invoked method throwing an exception.

Definition at line 20 of file AsyncParameters.cs.

◆ OnProgressUpdate

Action<StatusResponse> AnalyzeRe.LargeDataUpload.AsyncParameters.OnProgressUpdate

Action to invoke every time a chunk is uploaded. Callback is supplied with a double indicating progress towards completion (from 0 to 1).

Definition at line 24 of file AsyncParameters.cs.

◆ OnSuccess

Action<StatusResponse> AnalyzeRe.LargeDataUpload.AsyncParameters.OnSuccess

Action to invoke upon successful completion of the large upload.

Definition at line 15 of file AsyncParameters.cs.

Property Documentation

◆ binary_yelt_options

BinaryYELTUploadOptions AnalyzeRe.LargeDataUpload.Parameters.binary_yelt_options
getsetinherited
Initial value:
=
BinaryYELTUploadOptions.AutomaticBinaryConversionAndCompression
BinaryYELTUploadOptions
Options for controlling whether to automatically convert a binary YELTLossSet's data to the binary fo...
Definition Parameters.cs:67

Options for controlling whether to automatically convert a binary YELTLossSet's data to the binary format during upload. By default, the assumption is that csv data was provided and all conversion should be done on your behalf. Set to None if the file you're uploading is already in binary format and gzipped.

Definition at line 82 of file Parameters.cs.

◆ chunk_timeout

int AnalyzeRe.LargeDataUpload.Parameters.chunk_timeout = 1000 * 60
getsetinherited

The timeout (in milliseconds) for a single chunk upload to complete. Default: 60,000 ms (1 minute).

Definition at line 41 of file Parameters.cs.

◆ commit_polling_options

PollingOptions AnalyzeRe.LargeDataUpload.Parameters.commit_polling_options = DefaultCommitPollingOptions
getsetinherited

Determines how the system polls the status of the data file while waiting for the data to be processed. By default, these options use exponential back-off and will timeout if the system hasn't processed the file in 3,600,000 ms (1 hour).

Definition at line 47 of file Parameters.cs.

◆ Default

new AsyncParameters AnalyzeRe.LargeDataUpload.AsyncParameters.Default
staticget

Get a new set of Default Async Large Data Upload Parameters.

Definition at line 10 of file AsyncParameters.cs.

◆ DefaultCommitPollingOptions

PollingOptions AnalyzeRe.LargeDataUpload.Parameters.DefaultCommitPollingOptions
staticgetinherited

The default PollingOptions used when none are specified in a AnalyzeRe.LargeDataUpload.Parameters instance. By default, limits the maximum polling time to 1 hour. All other parameters use the default values for PollingOptions.

Definition at line 12 of file Parameters.cs.

◆ enable_compression

bool AnalyzeRe.LargeDataUpload.Parameters.enable_compression
getsetinherited

Whether to compress (gzip) the data during upload. (Default: False) Set to true if the files you're uploading are quite large and/or you wish to decrease network usage. In practice, if your upload speed is in excess of 5 MiBps, the overhead of compression and decompression on the server outweighs the reduction in uploaded data.

Definition at line 59 of file Parameters.cs.

◆ handle_existing_upload_strategy

HandleUploadInSessionStrategy AnalyzeRe.LargeDataUpload.Parameters.handle_existing_upload_strategy
getsetinherited
Initial value:
=
HandleUploadInSessionStrategy
Behaviour to use if an existing upload is already in progress for an endpoint.

The HandleUploadInSessionStrategy to employ of an existing upload session is already in progress. Default: RaiseError.

Definition at line 51 of file Parameters.cs.

◆ max_chunk_size

int AnalyzeRe.LargeDataUpload.Parameters.max_chunk_size = 16 * 1024 * 1024
getsetinherited

The maximum size of a single uploaded chunk in bytes. Default: 2^24 bytes (16 Megabytes).

We noted that chunk processing has n^2 complexity, where n is the chunk size. Chunk of 128Mb can result in timeout, and chunk size of 32-64Mb is very slow.

Definition at line 33 of file Parameters.cs.

◆ max_retries_per_chunk

int AnalyzeRe.LargeDataUpload.Parameters.max_retries_per_chunk = 3
getsetinherited

The maximum number of retries for a failed chunk upload before an error is thrown. Default: 3 retries.

Definition at line 37 of file Parameters.cs.

◆ min_chunk_size

int AnalyzeRe.LargeDataUpload.Parameters.min_chunk_size = 4 * 1024 * 1024
getsetinherited

The minimum size of a single uploaded chunk in bytes. Set this larger if there is enough latency between the server that smaller chunk requests decrease your throughput. Set this smaller if your upload stream is expensive to produce and you want to reduce the amount of time before the first chunk gets sent over the network layer. Default: 2^22 bytes (4 Megabytes).

Definition at line 27 of file Parameters.cs.


The documentation for this class was generated from the following file: