[Everyone] DAP data access

[Lawrence.Toomey at csiro.au]

Hi everyone,


I am writing this email in the hope to consolidate feedback from your experiences with accessing large pulsar data collections from CSIRO's Data Access Portal (https://data.csiro.au/dap). Please forward this email on to others who may be accessing data in this way.


Access of large data collections (in order of ~1.5 TB each) from the DAP present not only challenges to network infrastructure and logistics, but are likely to have the most impact on user experience.


However, given data volumes are already rising with the arrival of the UWL receiver (up to ~30 TB/day), accessing any data from the Parkes radio telescope will be a significant challenge, hence the importance of working on a better solutions.


We will need to begin to focus on the concept of 'bringing the user to the data', not the data to the user. With this in mind I am implementing a suite of virtual machines for cloud-based data access and/or pulsar data processing.


So in an effort to ensure that you gain access to your data in a timely manner, please send me as much feedback as you are able. For example, collating details such as the following would be extremely useful:

  *   ?the ability to find your data (e.g. whether you were able to search and find the data you needed)
  *   data availability timeframe (e.g. the time between requesting the data and retrieving the data)
  *   download speed (e.g. MB/s)
  *   download problems (e.g. timeouts, permissions)
  *   protocols used for downloading (e.g. wget, sftp, rsync)
  *   screenshots of errors


Thanks for all your help with this. If you have problems accessing your data from DAP, please do not hesitate to ask me for help.


Documentation on DAP data access is also available at the following link:

https://confluence.csiro.au/display/daphelp/Access+or+Download+Data




Regards,




Lawrence Toomey
Research Technician: Pulsar data archives
CSIRO Astronomy and Space Science (CASS)
Marsfield, NSW 2122

Office hours: Wed, Thurs, Fri
                      10am - 2pm

Phone: +61293724572
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pulsarastronomy.net/pipermail/everyone_lists.pulsarastronomy.net/attachments/20180911/9873d37e/attachment.html>