Downloading Data

Downloading Data

gsqlcmd allows downloading files and making HTTP requests.

For example:

gsqlcmd download https://www.nasdaq.com/symbol/AAPL/dividend-history aapl.htm

You may use a mask for output files if the URL contains a detected file name.

For example:

gsqlcmd download https://www.gsqlcmd.com/sitemap.xml *.*

/Check Option

You may use the /check option to test return results.

For example:

gsqlcmd download http://www.gsqlcmd.com/ /check

The result shows page redirect:

http://www.gsqlcmd.com/ 301 https://www.gsqlcmd.com/

Automating

You may use a simple or headered task file to download a list of URLs.

For example, you may place URLs to download in a file like sitemap.txt and use the command:

gsqlcmd download @sitemap.txt *.*

Also, you may omit @ in the task file name in the download mode.

For example:

gsqlcmd download sitemap.txt *.*

Below is a simple batch file that allows downloading sitemap.xml, converting it to sitemap.txt, and downloading its pages:

@echo off

gsqlcmd download https://www.gsqlcmd.com/sitemap.xml *.*

gsqlcmd convert "SELECT loc FROM sitemap.xml" sitemap.txt /noHeaders

gsqlcmd download @sitemap.txt *.*

You may even simplify the batch using a pipe:

gsqlcmd convert "SELECT loc FROM https://www.gsqlcmd.com/sitemap.xml" /noHeaders | gsqlcmd download @ *.*

You may this trick to check pages with the /check option:

gsqlcmd convert "SELECT loc FROM https://www.gsqlcmd.com/sitemap.xml" /noHeaders | gsqlcmd download @ *.* /check

Downloading Multiple Pages

gsqlcmd detects several schemes of the next page URLs.

In such cases, you may use the /pages option to download multiple pages.

For example, you use the following command to download option chains from Yahoo Finance:

gsqlcmd download "https://query1.finance.yahoo.com/v7/finance/options/AAPL" *.json /pages=20

GET, POST, PUT, PATCH, DELETE, HEAD

The download mode has the /method option with the following values:

AUTO, GET, POST, PUT, PATCH, MERGE, DELETE, HEAD

Here is an example of the DELETE method use:

gsqlcmd download https://localhost/crm/contacts?id=1 /method=DELETE

You may use the HTTP request method names as synonyms of the download mode omitting the /method option.

For example:

gsqlcmd DELETE https://localhost/crm/contacts?id=1

Using Authentication

Use the auth option to specify the authentication method.

gsqlcmd supports the following methods: Basic, Windows, Forms, OAuth1, and OAuth2.

For example:

gsqlcmd download https://localhost/crm/contacts /auth=Forms /username=user /password=pass

Additional Notes

See also Web Request Options.

This website is using cookies. By continuing to browse, you give us your consent to our use of cookies as explained in our Cookie Policy.