Redefining Data Download: How Database Mirroring Greatly Resolved Genomics Data Download Issue for Genome Institute of Singapore
7 Nov 2022 – Researchers from Genome Institute of Singapore (GIS) encountered problems with downloading genomics data directly from NIH’s NCBI FTP database. GIS researchers tried for more than 2 weeks but were still unable to download the desired genomics data successfully. They found that downloading data directly from NIH’s USA servers resulted in unacceptably slow speeds and erratic behaviour such as random aborts and incorrect file sizes. Besides experiencing latency during the downloading process, these downloaded files either failed or had incorrect file sizes.
Instead of wasting additional time and effort trying to download from NIH’s NCBI FTP database, GIS approached SingAREN to help with downloading genomics data. SingAREN recommended GIS to download genomics data from SingAREN Database Mirror (ftp://dbmirror.singaren.net.sg/) as it has a daily updated copy of the NCBI database. GIS could successfully download the genomic data directly using the SingAREN Database Mirroring service, reporting increase in transfer speed and reliability by several orders of magnitude.
“What took the researchers more than 2 weeks to download from NIH’s NCBI FTP database was successfully completed within a few minutes by SingAREN Database Mirroring. In contrast to previously incomplete downloaded files, SingAREN Database Mirroring has increased the reliability to 100%. We are very impressed and our users are very happy and grateful.” said Lim Moh Hui, Senior Systems Engineer II, Genome Institute of Singapore.
GIS researchers now regularly use SingAREN Database Mirroring to download genomics data. The Database Mirroring service enables high speed data downloads of major overseas scientific databases for Singapore’s research and education community, through mirroring the data in Singapore. This free service is available to the public. If you are interested in using SingAREN Database Mirroring, please visit ftp://dbmirror.singaren.net.sg. For requests to add databases to Database Mirroring, please submit your requests via the form at https://www.singaren.net.sg/contact-us/.
Written by Linda Lim
<a href=”https://www.freepik.com/free-vector/abstract-dna-background_4470794.htm#query=dna&from_query=dna%20data&position=0&from_view=search&track=sph”>Image by pikisuperstar</a> on Freepik