Paginated Downloads

Chunk-wise downloads of search results

Throught its Search Samples page Progenetix has always offered options to download search results (biosamples, variants) in different formats (JSON, tab-delimited tables, pgxseg files ...). However, especially for large results with thousands of samples and potentially millions of variants this led to inconsistent behaviour e.g. time-outs or dropped connections.

Now, API responses are capped through the limit parameter to default "sensible" values which, however, can be adjusted for systematic data access & retrieval. This functionality is also implemented in the sample search form, allowing e.g. the limited retrieval of a subset of samples from large or general cancer types, or the "paging" through consecutive sample groups for partitioned data retrieval.

We have recently worked on implementing a cleaner paging method through which data from Progenetix queries can be retrieved in consistent "parcels", using handover links. Technically, the ids of all items matched by the original query are stored internally and retrieved by their index ranges (e.g. samples 101-200) in several pre-defined formats.

Please see the Beacon framework documentation for a detailed description of the parameters.