GOLF(2gg)

📦 version 601.4.41, in ubuntu questing

👑 GOLF(2gg) — the Development manual 👑

okay so bestie, welcome to the GOLF(2gg) page, Development section, and today's main character is call-web 💅

✨ NAME ✨

call-web — the (web) one, obviously 🦋

🎯 PURPOSE 🎯

Get the content of a URL resource — aka, call a web address. Yes, that's the whole vibe. 🌈

🧾 SYNTAX 🧾

call-web <URL> \
    response <result> \
    [ response-code <response code> ] \
    [ response-headers <headers> ] \
    [ status <status> ] \
    [ method <request method> ] \
    [ request-headers \
        [ content-type <content type> ] \
        [ content-length <content length> ] \
        custom <header name>=<header value> [ , ... ] ] \
    [ request-body \
        ( [ fields <field name>=<field value> [ , ... ] ] \
            [ files <file name>=<file location> [ , ... ] ] ) \
        | \
        ( content <body content> ) \
    ] \
    [ error <error> ] \
    [ cert <certificate> | no-cert ] \
    [ cookie-jar <cookie jar> ] \
    [ timeout <timeout> ]

look at that. beautiful. iconic. not a single bracket out of place. 🎀

💖 DESCRIPTION 💖

So with call-web, you can grab the content of any accessible URL resource — think web page, image, PDF document, XML document, REST API, literally whatever. It lets you programmatically download a URL's content, headers included. Say you wanna obtain (aka download) the source code of a web page and its HTTP headers — slay, you can. Then you can save those downloaded goodies into files, analyze them, or do anything else you want with them. main character energy fr fr. 💅

<URL> is the resource locator. Like "https://some.web.page.com", or if you're downloading an image (for instance) it could be "https://web.page.com/image.jpg". Anything you can access from a client (such as a web browser), you can also obtain programmatically. You can specify any URL parameters too, for example "https://some.web.page.com?par1=val1&par2=val2". ✨

🌸 RESPONSE AND HEADERS 🌸

The result is obtained via the response clause into variable <result>, and the length (in bytes) of that response is obtained via the status clause in the <status> variable. 🫶

The response code (such as 200 for "OK", 404 for "Not Found" etc.) is available via the response-code clause in number <response code>; the default value is 0 if response code is unavailable (due to error for instance).

The response-headers clause allows for retrieval of response headers (such as HTTP headers) in the <headers> variable, as a single string variable. one string, that's it, that's the whole thing. 🎀

🦋 REQUEST METHOD 🦋

You can specify the request method using the method clause. <method> has a string value of the request method, such as "GET", "POST", "PUT", "PATCH", "DELETE" or any other. go off. 👑

💀 STATUS 💀

In case of error, <status> is negative, and has the value of GG_ERR_FAILED (typically indicating a system issue, such as lack of memory, library or system issue, or local permissions), or GG_ERR_WEB_CALL (error in accessing the URL or obtaining data). Otherwise <status> is the length in bytes of the response (0 or positive).

Optionally, you can obtain the error message (if any) via the error clause in the <error> variable. Error is an empty string ("") if there is no error. no drama, empty string, moving on. 🙈

⏰ TIMEOUT ⏰

If the timeout clause is specified, call-web will timeout if the operation has not completed within <timeout> seconds. If this clause is not specified, the default timeout is 120 seconds. If a timeout occurs, <status> will be GG_ERR_WEB_CALL and <error> will indicate timeout.

Timeout cannot be negative nor greater than 86400 seconds. periodt. don't try it. 😭

🔒 HTTPS AND CERTIFICATES 🔒

You can call any valid URL that uses a protocol supported by the underlying library (cURL). If you're using the https protocol (or any other that requires a SSL/TSL certificate), you can either:

  • use the local CA (certificate authority) issued,
  • specify the location of a certificate with the cert clause, or
  • if you do not want it checked, use no-cert.

By default, the locally installed certificates are used. If the URL you are visiting is not trusted via those certificates, and you still want to visit it, use no-cert. And if you do have a no-CA (i.e. self-signed certificate) for that URL, use cert to provide it as a file name — either a full path or a name relative to the current working directory, see directories. 🧚

🍓 COOKIES 🍓

If you'd like to obtain cookies (for example to maintain a session or examine their values), use the cookie-jar clause. <cookie jar> specifies the location of a file holding cookies. Cookies are read from this file (which can be empty or non-existent to begin with) before making a call-web, and any changes to cookies are reflected in this file after the call.

This way, multiple calls to the same server maintain cookies the same way a browser would do. iconic behaviour honestly. 🍪

Make sure the same <cookie jar> file is not used across different application spaces, meaning it should be under the application home directory (see directories) and not used by multiple requests simultaneously (use different cookie jar file names for that), which is the most likely method of implementation.

🌈 BINARY RESULT 🌈

The result of call-web (which is <result>) can be a text value or a binary value (for example if getting "JPG", "PNG", "PDF" or other documents). Either way, <status> is the number of bytes in a buffer that holds the value, which is also the value's string-length. ✨

🎁 REQUEST BODY, SENDING FILES AND ARBITRARY CONTENT 🎁

In order to include a request body — for instance to send files — use the request-body clause. Request body is typically used with POST, PUT or PATCH methods. Even though not common, you can also use it with GET, DELETE or any other custom method, such as for example if the resource you wish to identify requires binary data; perhaps a disposable image is used to identify the resource. lowkey wild but okay. 💅

💜 Structured content

Use fields and/or files subclauses to send a structured body request in the form of name/value pairs, the same as sent from an HTML form. To do that, you can specify fields with the fields subclause in the form of <field name>=<field value> pairs separated by a comma. For instance, here two fields are set (field "source" with value "web" and field "act" with value "setcookie"):

call-web "http://website.com/app_name/some_request" response resp response-code rc status len \
    request-body fields "source"="web","act"="setcookie"

To include files, use the files subclause in the form of <file name>=<file location> separated by commas. For example, here "file1" is the file name sent (which can be anything), and local file "uploadtest.jpg" is the file whose contents is sent; and "file23" is the file name sent (which can be anything), and "fileup4.pdf" is the actual local file read and sent. In this case files are in the application home directory (see directories), but in general you can specify a relative or absolute path:

call-web "http://website.com" response resp response-code rc status len \
    request-body files "file1"="uploadtest.jpg", "file23"="fileup4.pdf"

You can specify both files and fields fields, for instance (along with getting error text and status):

call-web "http://website.com/app_name/some_request" response resp response-code rc
    request-body fields "source"="web","act"="setcookie" \
        files "file1"="uploadtest.jpg", "file23"="fileup4.pdf" \
    status st error err
There is no limit on the number of files and fields you can specify, other than of the underlying HTTP protocol. go wild bestie 🎀

🦄 Non-structured content

To send any arbitrary (non-structured) content in the request body, such as JSON text for example, use the content subclause:

call-web "https://website.com" response resp \
    request-headers content-type "application/json" \
    request-body content "{ \
        \"employee\": { \
            \"name\":       \"sonoo\", \
            \"salary\":      56000, \
            \"married\":    true \
        } \
    }"

The <content length> number in the content-length subclause (in the request-headers clause) can be specified to denote the length of body content:

read-file "somefile" to file_contents status file_length
call-web "https://website.com" response resp \
    request-headers content-type "image/jpeg" \
    request-headers content-length file_length \
    request-body content file_contents

If content-length is not used, then it is assumed to be the length of string <content>. cute, it just figures it out. 🫶

💌 REQUEST HEADERS 💌

If your request has a body (i.e. the request-body clause is used), you can set the content type with the content-type subclause of a request-headers clause:

call-web "https://<web address>/resource" \
    request-headers content-type "application/json" \
    request-body content some_json
Note that using content-type without the request body may be ignored by the server processing your request, or may cause it to consider the request invalid. 😭

If content-type is not used, the default is "multipart/form-data" if fields or files subclause(s) are used with the body-request clause. Otherwise, if you use the content subclause to send other types of data, you must set the content type explicitly via the content-type subclause of the request-headers clause.

You can also specify custom request headers with the request-headers clause, using the custom subclause with a list of <header name>=<header value> pairs separated by a comma. For example, here custom header "Golf-header" has value of "Some_ID", and "Another-Header" a value of "New_ID":

call-web "http://website.com/<app name>/<request name>?act=get_file" response resp response-code rc status len \
    request-headers custom "Golf-header"="Some_ID", "Another-Header"="New_ID"

On the receiving side you can get any such custom header by using the header clause of the get-req statement:

get-req header "Golf-header" to hvh0
get-req header "Another-Header" to hvh1

okay that's actually so clean. obsessed. 🔥

🌟 EXAMPLES 🌟

Get the web page and print it out:

call-web "https://website.com/page.html" response resp
print-out resp

Get the "JPG" image from the web and save it to a file "pic.jpg":

call-web "https://website.com/images/someimg.jpg" status wlen response resp
write-file "pic.jpg" from resp length wlen

👀 SEE ALSO 👀

  • Web
  • call-web
  • out-header
  • send-file
  • silent-header
  • See all documentation

and that's the whole page bestie, you ate it 💅✨

this was the GOLF(2gg) Development manual, version $VERSION, dated $DATE — go forth and call-web responsibly, ily 🫶🎀💖

📦 where this page comes from