numpy.DataSource

class numpy.DataSource(destpath='.')

A generic data source file (file, http, ftp, ...).

DataSources could be local files or remote files/URLs. The files may also be compressed or uncompressed. DataSource hides some of the low-level details of downloading the file, allowing you to simply pass in a valid file path (or URL) and obtain a file object.

Methods:

• exists : test if the file exists locally or remotely
• abspath : get absolute path of the file in the DataSource directory
• open : open the file

Example URL DataSource:

# Initialize DataSource with a local directory, default is os.curdir.
ds = DataSource('/home/guido')

# Open remote file.
# File will be downloaded and opened from here:
#     /home/guido/site/xyz.txt
ds.open('http://fake.xyz.web/site/xyz.txt')

Example using DataSource for temporary files:

# Initialize DataSource with 'None' for the local directory.
ds = DataSource(None)

# Open local file.
# Opened file exists in a temporary directory like:
#     /tmp/tmpUnhcvM/foobar.txt
# Temporary directories are deleted when the DataSource is deleted.
ds.open('/home/guido/foobar.txt')

Notes:

BUG : URLs require a scheme string (‘http://‘) to be used.

www.google.com will fail.

>>> repos.exists('www.google.com/index.html')
False

>>> repos.exists('http://www.google.com/index.html')
True