Downloading Data with gsqlcmd
The gsqlcmd tool allows you to download files and make HTTP requests.
For example, to download a file from a URL, use:
gsqlcmd download https://www.nasdaq.com/symbol/AAPL/dividend-history aapl.htm
You can use a wildcard for output files if the URL contains a recognizable file name:
gsqlcmd download https://www.gsqlcmd.com/sitemap.xml .
/Check Option
Use the /check option to test the return results.
For example:
gsqlcmd download http://www.gsqlcmd.com/ /check
The result will show any page redirects:
http://www.gsqlcmd.com/ 301 https://www.gsqlcmd.com/
Automating
You can create a simple or headered task file to download a list of URLs.
For instance, if you have a file named sitemap.txt containing URLs, you can run:
gsqlcmd download @sitemap.txt .
You can also omit the @ in the task file name when using the download mode:
gsqlcmd download sitemap.txt .
Here’s a simple batch file that downloads sitemap.xml, converts it to sitemap.txt, and then downloads its pages:
@echo off gsqlcmd download https://www.gsqlcmd.com/sitemap.xml . gsqlcmd convert "SELECT loc FROM sitemap.xml" sitemap.txt /noHeaders gsqlcmd download @sitemap.txt .
You can simplify the batch process using a pipe:
gsqlcmd convert "SELECT loc FROM https://www.gsqlcmd.com/sitemap.xml" /noHeaders | gsqlcmd download @ .
You can also check pages using the /check option:
gsqlcmd convert "SELECT loc FROM https://www.gsqlcmd.com/sitemap.xml" /noHeaders | gsqlcmd download @ . /check
Downloading Multiple Pages
gsqlcmd can detect various schemes for next page URLs. In such cases, use the /pages option to download multiple pages.
For example, to download option chains from Yahoo Finance, run:
gsqlcmd download "https://query1.finance.yahoo.com/v7/finance/options/AAPL" *.json /pages=20
GET, POST, PUT, PATCH, DELETE, HEAD
The download mode includes the /method option, which accepts the following values:
- AUTO
- GET
- POST
- PUT
- PATCH
- MERGE
- DELETE
- HEAD
Here’s an example using the DELETE method:
gsqlcmd download https://localhost/crm/contacts?id=1 /method=DELETE
You can also use HTTP request method names as synonyms for the download mode with the specified /method option:
gsqlcmd DELETE https://localhost/crm/contacts?id=1
Using Authentication
Use the /auth option to specify the authentication method, along with method-specific options to provide authentication values.
For example:
gsqlcmd download https://localhost/crm/contacts /auth=Forms /username=user /password=pass
gsqlcmd supports the following authentication methods: Basic, Windows, Forms, OAuth1, and OAuth2.
For more details, see HTTP Authentication.
Additional Notes
Refer to Web Request Options for more information.