PYTHON-SWIFTCLIENT(1)
💅 PYTHON-SWIFTCLIENT(1) 💅
okay bestie, welcome to the python-swiftclient manual page — this is python-swiftclient 4.10.0, and it's literally giving command line tool plus Python APIs. stay with me, it's gonna slay. 💖
💖 NAME 💖
python-swiftclient — python-swiftclient 4.10.0. periodt.
🌸 INTRODUCTION 🌸
💫 Introduction 💫
Bestie, the python-swiftclient project is a command line tool plus two separate APIs for accessing swift programmatically. The first problem you gotta solve is picking the right method for your use case. lowkey the whole vibe depends on it. ✨
🧭 Where to Start? 🧭
The python-swiftclient project comprises a command line tool and two separate APIs for accessing swift programmatically. Choosing the most appropriate method for a given use case is the first problem a user needs to solve.
🎬 Use Cases 🎬
Alongside the command line tool, python-swiftclient includes two levels of API:
- A low level client API — simple Python wrappers around the various authentication mechanisms and the individual HTTP requests. 🐍
- A high level service API — methods for performing common operations in parallel on a thread pool. 🧵
Example use cases:
- Uploading and retrieving data — use the command line tool if you're simply uploading and downloading files and directories to and from your filesystem. The command line tool can be integrated into a shell script to automate tasks. 🗂️
- Integrating into an automated Python workflow — use the
SwiftServiceAPI to perform operations offered by the CLI if your use case requires integration with a Python-based workflow. This method offers greater control and flexibility over individual object operations, such as the metadata set on each object. TheSwiftServiceclass provides methods to perform multiple sets of operations against a swift object store using a configurable shared thread pool. A single instance of theSwiftServiceclass can be shared between multiple threads in your own code. 💻 - Developing an application in Python to access a swift object store — use the
SwiftServiceAPI to develop Python applications that use swift to store and retrieve objects. ASwiftServiceinstance provides a configurable thread pool for performing all operations supported by the CLI. 🐍✨ - Fine-grained control over threading or the requests being performed — use the
ConnectionAPI if your use case requires fine grained control over advanced features or you wish to use your own existing threading model. Examples of advanced features requiring the use of theConnectionAPI include creating an SLO manifest that references already existing objects, or fine grained control over the query strings supplied with each HTTP request. 🔧
⚠️ Important considerations ⚠️
This section covers some important considerations, helpful hints, and things to avoid when integrating an object store into your workflow.
🚫 An object store is not a filesystem 🚫
It cannot be stressed enough that your usage of the object store should reflect the proper use case, and not treat the storage like a traditional filesystem. There are two main restrictions to bear in mind when designing an application that uses an object store:
- You cannot rename objects. Due to the fact that the name of an object is one of the factors that determines where the object and its replicas are stored, renaming would require multiple copies of the data to be moved between physical storage devices. If you want to rename an object you must upload to the new location, or make a server side copy request to the new location, and then delete the original. 😭
- You cannot modify objects. Objects are stored in multiple locations and are checked for integrity based on the MD5 sum calculated during upload. In order to modify the contents of an object, the entire desired contents must be re-uploaded. In certain special cases it is possible to work around this restriction using large objects, but no general file-like access is available to modify a stored object. 💀
🔒 Objects cannot be locked 🔒
There is no mechanism to perform a combination of reading the data/metadata from an object and writing an update to that data/metadata in an atomic way. Any user with access to a container could update the contents or metadata associated with an object at any time.
Workflows that assume that no updates have been made since the last read of an object should be discouraged. Enabling a workflow of this type requires an external object locking mechanism and/or cooperation between all clients accessing the data. 🫶
🛠️ DEVELOPER DOCUMENTATION 🛠️
🫶 So You Want to Contribute... 🫶
For general information on contributing to OpenStack, please check out the contributor guide to get started. It covers all the basics that are common to all OpenStack projects: the accounts you need, the basics of interacting with our Gerrit review system, how we communicate as a community, etc.
The python-swiftclient is maintained by the OpenStack Swift project. To understand our development process and how you can contribute to it, please look at the Swift project's general contributor's page: http://docs.openstack.org/swift/latest/contributor/contributing.html
💻 CLI 💻
The swift tool is a command line utility for communicating with an OpenStack Object Storage (swift) environment. It allows one to perform several types of operations. 👑
For help on a specific swift command, enter:
$ swift COMMAND --help🚀 swift usage 🚀
Usage: swift [--version] [--help] [--os-help] [--verbose]
[--debug] [--info] [--quiet] [--auth <auth_url>]
[--auth-version <auth_version> |
--os-identity-api-version <auth_version> ]
[--user <username>]
[--key <api_key>] [--retries <num_retries>]
[--os-username <auth-user-name>] [--os-password <auth-password>]
[--os-user-id <auth-user-id>]
[--os-user-domain-id <auth-user-domain-id>]
[--os-user-domain-name <auth-user-domain-name>]
[--os-tenant-id <auth-tenant-id>]
[--os-tenant-name <auth-tenant-name>]
[--os-project-id <auth-project-id>]
[--os-project-name <auth-project-name>]
[--os-project-domain-id <auth-project-domain-id>]
[--os-project-domain-name <auth-project-domain-name>]
[--os-auth-url <auth-url>] [--os-auth-token <auth-token>]
[--os-storage-url <storage-url>] [--os-region-name <region-name>]
[--os-service-type <service-type>]
[--os-endpoint-type <endpoint-type>]
[--os-cacert <ca-certificate>] [--insecure]
[--os-cert <client-certificate-file>]
[--os-key <client-certificate-key-file>]
[--no-ssl-compression]
<subcommand> [--help] [<subcommand options>]Subcommands:
delete— deletes a container or objects within a container. 🗑️download— downloads objects from containers. ⬇️list— lists the containers for the account or the objects for a container. 📋post— updates meta information for the account, container, or object; creates containers if not present. 📝copy— copies object, optionally adds meta. 📄stat— displays information for the account, container, or object. 📊upload— uploads files or directories to the given container. ⬆️capabilities— list cluster capabilities. 🧠tempurl— create a temporary URL. 🔗auth— display auth related environment variables. 🔐
⚙️ swift optional arguments ⚙️
--version— show program's version number and exit.-h,--help— show this help message and exit.--os-help— show OpenStack authentication options.-v,--verbose— print more info.--debug— show the curl commands and results of all http queries regardless of result status.--info— show the curl commands and results of all http queries which return an error.-q,--quiet— suppress status output.-A AUTH,--auth=AUTH— URL for obtaining an auth token.-V AUTH_VERSION,--auth-version=AUTH_VERSION,--os-identity-api-version=AUTH_VERSION— specify a version for authentication. Defaults toenv[ST_AUTH_VERSION],env[OS_AUTH_VERSION],env[OS_IDENTITY_API_VERSION]or1.0.-U USER,--user=USER— user name for obtaining an auth token.-K KEY,--key=KEY— key for obtaining an auth token.-R RETRIES,--retries=RETRIES— the number of times to retry a failed connection.--insecure— allow swiftclient to access servers without having to verify the SSL certificate. Defaults toenv[SWIFTCLIENT_INSECURE](set to'true'to enable). 🚨--no-ssl-compression— this option is deprecated and not used anymore. SSL compression should be disabled by default by the system SSL library. 💀--prompt— prompt user to enter a password which overrides any password supplied via--key,--os-passwordor environment variables. 🙈
🔐 Authentication 🔐
This section covers the options for authenticating with a swift object store. The combinations of options required for each authentication version are detailed below, but are just a subset of those that can be used to successfully authenticate. These are the most common and recommended combinations.
You should obtain the details of your authentication version and credentials from your storage provider. These details should make it clearer which of the authentication sections below are most likely to allow you to connect to your storage account. 🌈
🪄 Keystone v3 🪄
swift --os-auth-url https://api.example.com:5000/v3 --auth-version 3 \
--os-project-name project1 --os-project-domain-name domain1 \
--os-username user --os-user-domain-name domain1 \
--os-password password list
swift --os-auth-url https://api.example.com:5000/v3 --auth-version 3 \
--os-project-id 0123456789abcdef0123456789abcdef \
--os-user-id abcdef0123456789abcdef0123456789 \
--os-password password listManually specifying the options above on the command line can be avoided by setting the following combinations of environment variables:
ST_AUTH_VERSION=3
OS_USERNAME=user
OS_USER_DOMAIN_NAME=domain1
OS_PASSWORD=password
OS_PROJECT_NAME=project1
OS_PROJECT_DOMAIN_NAME=domain1
OS_AUTH_URL=https://api.example.com:5000/v3
ST_AUTH_VERSION=3
OS_USER_ID=abcdef0123456789abcdef0123456789
OS_PASSWORD=password
OS_PROJECT_ID=0123456789abcdef0123456789abcdef
OS_AUTH_URL=https://api.example.com:5000/v3🪄 Keystone v2 🪄
swift --os-auth-url https://api.example.com:5000/v2.0 \
--os-tenant-name tenant \
--os-username user --os-password password listST_AUTH_VERSION=2.0
OS_USERNAME=user
OS_PASSWORD=password
OS_TENANT_NAME=tenant
OS_AUTH_URL=https://api.example.com:5000/v2.0Like, why type all those flags every single time when your shell can just remember them for you? Bestie. Set the vars, live your life. 💅
🌸 Legacy auth systems 🌸
Okay so here's the tea — you can point swift at literally any number of other auth systems, and we're simply not covering them in this doc. Not our lane. If your storage provider isn't using Keystone to hand out access tokens, go ask them what options you need, because they know and we don't. Best guess? It's gonna look something like this:
swift -A https://api.example.com/v1.0 -U user -K api_key listAnd again, the group-chat hack: you can dodge typing all of that on the command line by setting these environment variables instead:
ST_AUTH_VERSION=1.0
ST_AUTH=https://api.example.com/v1.0
ST_USER=user
ST_KEY=keyThere's also a possibility that you're on a completely different auth system, and in that case swiftclient literally cannot request a token for you — it doesn't have the range, sorry bestie. So you go make the authentication request yourself, separately, and then access your storage using the token and storage URL options right here:
swift --os-auth-token 6ee5eb33efad4e45ab46806eac010566 \
--os-storage-url https://10.1.5.2:8080/v1/AUTH_ced809b6a4baea7aeab61a \
listNOTE: leftover environment variables are a super common source of confusion when authorization fails. Like, you'll be sitting there like "why is this not working 😭" and it's because a stale OS_* var from three projects ago is still hanging around in your shell. Do a little vibe check on your env, babes.
🎀 CLI commands 🎀
Auth 🫶
Usage: swift authDisplays your authentication variables in a shell-friendly format ✨. Wanna export the storage URL and auth token into OS_STORAGE_URL and OS_AUTH_TOKEN? Run swift auth. Wanna append it to a runcom file (like ~/.bashrc or /etc/profile) so you get automatic authentication forever and ever? Run swift auth -v -U test:tester -K testing. Iconic, honestly.
swift stat 🔍
Usage: swift stat [--lh] [--header <header:value>]
[<container> [<object>]]Displays information for the account, container, or object — whichever one you handed it (or none, if you handed it nothing). And in verbose mode it also shows you the storage URL and the authentication token. Nosy but useful 💖
Positional arguments:
- [container] — name of the container to stat from.
- [object] — name of the object to stat.
Optional arguments:
--lh— report sizes in human readable format, similar tols -lh.-H,--header <header:value>— adds a custom request header to use for stat.
swift list 🧾
Usage: swift list [--long] [--lh] [--totals] [--prefix <prefix>]
[--delimiter <delimiter>] [--header <header:value>]
[<container>]Lists the containers for the account, or the objects for a container. Super straightforward, no cap.
The -p <prefix> or --prefix <prefix> option only lists items beginning with that prefix. The -d <delimiter> or --delimiter <delimiter> option is for container listings only and rolls up items with the given delimiter — for what that actually means, go peek at the OpenStack Swift general documentation: https://docs.openstack.org/swift/latest/.
The -l and --lh options give you more detail, like ls -l and ls -lh, with the latter handing you sizes in human readable format (for example: 3K, 12M, etc). Heads up though — those last two switches use more overhead to retrieve all the displayed details, and that overhead is directly proportional to the number of containers or objects you're listing. So like, don't --lh a million objects and then act surprised 💀
Positional arguments:
- [container] — name of container to list object in.
Optional arguments:
-l,--long— long listing format, similar tols -l.--lh— report sizes in human readable format similar tols -lh.-t,--totals— used with-lor--lh, only report totals.-p <prefix>,--prefix <prefix>— only list items beginning with the prefix.-d <delim>,--delimiter <delim>— roll up items with the given delimiter. For containers only. See OpenStack Swift API documentation for what this means.-H,--header <header:value>— adds a custom request header to use for listing.
swift upload 🚀
Usage: swift upload [--changed] [--skip-identical] [--segment-size <size>]
[--segment-container <container>] [--leave-segments]
[--object-threads <thread>] [--segment-threads <threads>]
[--header <header>] [--use-slo] [--ignore-checksum]
[--object-name <object-name>]
<container> <file_or_directory> [<file_or_directory>] [...]Uploads the files and directories specified by the remaining arguments into the given container. Let's break the vibes down:
The -c or --changed option only uploads files that have changed since the last upload. The --object-name <object-name> option uploads a file and names the object <object-name>, or uploads a directory and uses <object-name> as the object prefix. And if the file name is -, the client reads content from standard input — in that case --object-name is required so it knows what to call the object, and no other files may be given. The -S <size> or --segment-size <size> and --leave-segments are options too (see --help for more).
Positional arguments:
<container>— name of container to upload to.<file_or_directory>— name of file or directory to upload. Specify multiple times for multiple uploads.
Optional arguments:
-c,--changed— only upload files that have changed since the last upload.--skip-identical— skip uploading files that are identical on both sides.-S,--segment-size <size>— upload files in segments no larger than<size>(in Bytes) and then create a "manifest" file that will download all the segments as if it were the original file. Big-file energy ✨--segment-container <container>— upload the segments into the specified container. If you don't specify one, the segments go into a<container>_segmentscontainer so they don't pollute the main<container>listings. Considerate of them, honestly.--leave-segments— indicates that you want the older segments of manifest objects left alone (in the case of overwrites).--object-threads <threads>— number of threads to use for uploading full objects. Default is10.--segment-threads <threads>— number of threads to use for uploading object segments. Default is10.-H,--header <header:value>— adds a customized request header. This option may be repeated. Example:-H "content-type:text/plain" -H "Content-Length: 4000".--use-slo— when used in conjunction with--segment-sizeit will create a Static Large Object instead of the default Dynamic Large Object.--object-name <object-name>— upload file and name object to<object-name>, or upload dir and use<object-name>as object prefix instead of the folder name.--ignore-checksum— turn off checksum validation for uploads.
swift post 📮
Usage: swift post [--read-acl <acl>] [--write-acl <acl>] [--sync-to <sync-to>]
[--sync-key <sync-key>] [--meta <name:value>]
[--header <header>]
[<container> [<object>]]Updates meta information for the account, container, or object depending on the arguments you gave it.
Now, real talk: if the container isn't found, swiftclient will create it automatically — but that is not true for accounts and objects. Containers also allow the -r <read-acl> (or --read-acl <read-acl>) and -w <write-acl> (or --write-acl <write-acl>) options. The -m or --meta option is allowed on accounts, containers and objects, and it defines the user metadata items to set in the form Name:Value. You can repeat this option. For example: post -m Color:Blue -m Size:Large.
For more information about ACL formats see the documentation: ACLs.
Positional arguments:
- [container] — name of container to post to.
- [object] — name of object to post.
Optional arguments:
-r,--read-acl <acl>— read ACL for containers. Quick summary of ACL syntax:.r:*,.r:-.example.com,.r:www.example.com,account1(v1.0 identity API only),account1:*,account2:user2(v2.0+ identity API).-w,--write-acl <acl>— write ACL for containers. Quick summary of ACL syntax:account1(v1.0 identity API only),account1:*,account2:user2(v2.0+ identity API).-t,--sync-to <sync-to>— Sync To for containers, for multi-cluster replication.-k,--sync-key <sync-key>— Sync Key for containers, for multi-cluster replication.-m,--meta <name:value>— sets a meta data item. This option may be repeated.
Example: -m Color:Blue -m Size:Large
-H,--header <header:value>— adds a customized request header. This option may be repeated.
Example: -H "content-type:text/plain" -H "Content-Length: 4000"
swift download 💾
Usage: swift download [--all] [--marker <marker>] [--prefix <prefix>]
[--output <out_file>] [--output-dir <out_directory>]
[--object-threads <threads>] [--ignore-checksum]
[--container-threads <threads>] [--no-download]
[--skip-identical] [--remove-prefix]
[--header <header:value>] [--no-shuffle]
[<container> [<object>] [...]]Downloads everything in the account (with --all), or everything in a container, or a list of objects — depends what you gave it. For a single object download you can use the -o <filename> or --output <filename> option to redirect the output to a specific file, or - to redirect it to stdout. The --ignore-checksum option turns off checksum validation. You can specify optional headers with the repeatable cURL-like option -H [--header <name:value>]. And --ignore-mtime ignores the x-object-meta-mtime metadata entry on the object (if present) and instead creates the downloaded files with fresh atime and mtime values. 🕰️
Positional arguments:
<container>— name of container to download from. To download a whole account, omit this and specify--all.<object>— name of object to download. Specify multiple times for multiple objects. Omit this to download all objects from the container.
Optional arguments:
-a,--all— indicates that you really want to download everything in the account. Like, really really.-m,--marker <marker>— marker to use when starting a container or account download.-p,--prefix <prefix>— only download items beginning with<prefix>-r,--remove-prefix— an optional flag for--prefix <prefix>, use this option to download items without<prefix>-o,--output <out_file>— got one single file to grab? This streams the output straight intoout_file. And if you pass"-"as the<out_file>, it redirects to stdout instead. sneaky but iconic ✨-D,--output-dir <out_directory>— an optional directory where all your objects get stored. By default everything gets recreated right in the current directory, so don't be surprised when files start appearing in your pwd like little gremlins 🫶--object-threads <threads>— how many threads to use for downloading objects. Default is10.--container-threads <threads>— how many threads for downloading containers. Default is10.--no-download— performs the download(s) but doesn't actually write anything to disk. it's giving "practice run" energy 💅-H,--header <header:value>— adds a customized request header to the query, like"Range"or"If-Match". This option may be repeated.
Example: --header "content-type:text/plain"
--skip-identical— skips downloading files that are identical on both sides. no redundant downloads in this house 🎀--ignore-checksum— turns off checksum validation for downloads.--no-shuffle— okay so by default, when you're downloading a complete account or container, the download order is randomised to reduce the load on individual drives when multiple clients are executed simultaneously to download the same set of objects (like, say, a nightly automated download script hitting multiple servers). Enable this option to submit download jobs to the thread pool in the order they're listed in the object store. controlled chaos vs. orderly chaos, your call 🔥
swift delete
Usage: swift delete [--all] [--leave-segments]
[--object-threads <threads>]
[--container-threads <threads>]
[--header <header:value>]
[<container> [<object>] [...]]Deletes everything in the account (with --all), or everything in a container, or a list of objects — depends on the arguments you give it. Segments of manifest objects will be deleted as well, unless you specify the --leave-segments option. it's a clean sweep bestie 🧹
Positional arguments:
[<container>]— name of container to delete from.[<object>]— name of object to delete. Specify multiple times for multiple objects.
Optional arguments:
-a,--all— delete all containers and objects. everything. gone. 💀--leave-segments— do not delete segments of manifest objects.-H,--header <header:value>— adds a custom request header to use for deleting objects or an entire container.--object-threads <threads>— number of threads to use for deleting objects. Default is10.--container-threads <threads>— number of threads to use for deleting containers. Default is10.
swift copy
Usage: swift copy [--destination </container/object>] [--fresh-metadata]
[--meta <name:value>] [--header <header>] <container>
<object> [<object>] [...]Copies an object to a new destination or adds user metadata to an object. Depending on the options supplied, you can preserve existing metadata — in contrast to the post command. The --destination option sets the copy target destination in the form /container/object. If you don't set it, the object will be copied onto itself, which is useful for adding metadata. You can use the -M or --fresh-metadata option to copy an object without existing user meta data, and the -m or --meta option to define user meta data items to set in the form Name:Value. You can repeat this option. For example: copy -m Color:Blue -m Size:Large. copy-paste but make it cloud ☁️💅
Positional arguments:
<container>— name of container to copy from.<object>— name of object to copy. Specify multiple times for multiple objects.
Optional arguments:
-d,--destination </container[/object]>— the container and name of the destination object. Name of destination object can be omitted, then will be same as name of source object. Supplying multiple objects and destination with object name is invalid. ⚠️-M,--fresh-metadata— copy the object without any existing metadata. If not set, metadata will be preserved or appended.-m,--meta <name:value>— sets a meta data item. This option may be repeated.
Example: -m Color:Blue -m Size:Large
-H,--header <header:value>— adds a customized request header. This option may be repeated. 🎀
Example: -H "content-type:text/plain" -H "Content-Length: 4000"
swift capabilities
Usage: swift capabilities [--json] [<proxy_url>]Displays cluster capabilities. The output includes the list of the activated Swift middlewares as well as relevant options for each ones. Additionally the command displays relevant options for the Swift core. If the proxy-url option is not provided, the storage URL retrieved after authentication is used as proxy-url. it's basically a vibe check for your whole cluster 👑
Optional positional arguments:
<proxy_url>— proxy URL of the cluster to retrieve capabilities.--json— print the cluster capabilities in JSON format.
swift tempurl
Usage: swift tempurl [--absolute] [--prefix-based]
<method> <seconds> <path> <key>Generates a temporary URL for a Swift object. 🌈
The method option sets an HTTP method to allow for this temporary URL — that's usually GET or PUT.
The time option sets the amount of time the temporary URL will be valid for. time can be specified as an integer, denoting the number of seconds from now on until the URL shall be valid; or, if --absolute is passed, the Unix timestamp when the temporary URL will expire. But beyond that, time can also be specified as an ISO 8601 timestamp in one of following formats:
- Complete date:
YYYY-MM-DD(e.g.1997-07-16) - Complete date plus hours, minutes and seconds:
YYYY-MM-DDThh:mm:ss(e.g.1997-07-16T19:20:30) - Complete date plus hours, minutes and seconds with UTC designator:
YYYY-MM-DDThh:mm:ssZ(e.g.1997-07-16T19:20:30Z)
Please be aware that if you don't provide the UTC designator (i.e.,Z) the timestamp is generated using your local timezone. If only a date is specified, the time part used will equal to00:00:00. no cap, timezones will humble you 😭
The path option sets the full path to the Swift object. Example: /v1/AUTH_account/c/o.
The key option is the secret temporary URL key set on the Swift cluster. To set a key, run swift post -m "Temp-URL-Key: <your secret key>".
To generate a prefix-based temporary URL use the --prefix-based option. This URL will contain the path to the prefix. Do not forget to append the desired objectname at the end of the path portion (and before the query portion) before sharing the URL. It is possible to use ISO 8601 UTC timestamps within the URL by using the --iso8601 option.
Positional arguments:
<method>— an HTTP method to allow for this temporary URL. Usually'GET'or'PUT'.<seconds>— the amount of time in seconds the temporary URL will be valid for; or, if--absoluteis passed, the Unix timestamp when the temporary URL will expire.<path>— the full path to the Swift object.
Example: /v1/AUTH_account/c/o or: http://saio:8080/v1/AUTH_account/c/o
<key>— the secret temporary URL key set on the Swift cluster. To set a key, run'swift post -m "Temp-URL-Key:b3968d0207b54ece87cccc06515a89d4"'
Optional arguments:
--absolute— interpret the<seconds>positional argument as a Unix timestamp rather than a number of seconds in the future.--prefix-based— if present, a prefix-based tempURL will be generated.
Examples
In this section we present some example usage of the swift CLI. To keep the examples as short as possible, these examples assume that the relevant authentication options have been set using environment variables. You can obtain the full list of commands and options available in the swift CLI by executing the following:
> swift --help
> swift <command> --helpSimple examples
List the existing swift containers:
> swift list
container_1Create a new container:
> swift post TestContainerUpload an object into a container:
> swift upload TestContainer testSwift.txt
testSwift.txtList the contents of a container:
> swift list TestContainer
testSwift.txtCopy an object to new destination:
> swift copy -d /DestContainer/testSwift.txt SourceContainer testSwift.txt
SourceContainer/testSwift.txt copied to /DestContainer/testSwift.txtDelete an object from a container:
> swift delete TestContainer testSwift.txt
testSwift.txtDelete a container:
> swift delete TestContainer
TestContainerDisplay auth related authentication variables in shell friendly format:
> swift auth
export OS_STORAGE_URL=http://127.0.0.1:8080/v1/AUTH_bf5e63572f7a420a83fcf0aa8c72c2c7
export OS_AUTH_TOKEN=c597015ae19943a18438b52ef3762e79Download an object from a container:
> swift download TestContainer testSwift.txt
testSwift.txt [auth 0.028s, headers 0.045s, total 0.045s, 0.002 MB/s]NOTE: To upload an object to a container, your current working directory must be where the file is located or you must provide the complete path to the file. In other words, the--object-name <object-name>is an option that will upload file and name object to<object-name>or upload directory and use<object-name>as object prefix. In the case that you provide the complete path of the file, that complete path will be the name of the uploaded object. 💖
For example:
> swift upload TestContainer /home/swift/testSwift/testSwift.txt
home/swift/testSwift/testSwift.txt
> swift list TestContainer
home/swift/testSwift/testSwift.txtMore complex examples
Swift has a single object size limit of 5GiB. In order to upload files larger than this, we must create a large object that consists of smaller segments. The example below shows how to upload a large video file as a static large object in 1GiB segments:
> swift upload videos --use-slo --segment-size 1G myvideo.mp4
myvideo.mp4 segment 8
myvideo.mp4 segment 4
myvideo.mp4 segment 2
myvideo.mp4 segment 7
myvideo.mp4 segment 0
myvideo.mp4 segment 1
myvideo.mp4 segment 3
myvideo.mp4 segment 6
myvideo.mp4 segment 5
myvideo.mp4This command will upload segments to a container named videos_segments, and create a manifest file describing the entire object in the videos container. For more information on large objects, see the documentation here.
> swift list videos
myvideo.mp4
> swift list videos_segments…and the segment list keeps rolling like this, bestie:
myvideo.mp4/slo/1460229233.679546/9341553868/1073741824/00000000
myvideo.mp4/slo/1460229233.679546/9341553868/1073741824/00000001
myvideo.mp4/slo/1460229233.679546/9341553868/1073741824/00000002
myvideo.mp4/slo/1460229233.679546/9341553868/1073741824/00000003
myvideo.mp4/slo/1460229233.679546/9341553868/1073741824/00000004
myvideo.mp4/slo/1460229233.679546/9341553868/1073741824/00000005
myvideo.mp4/slo/1460229233.679546/9341553868/1073741824/00000006
myvideo.mp4/slo/1460229233.679546/9341553868/1073741824/00000007
myvideo.mp4/slo/1460229233.679546/9341553868/1073741824/00000008First bestie, set the key, then generate a temporary URL for a Swift object:
> swift post -m "Temp-URL-Key:b3968d0207b54ece87cccc06515a89d4"
> swift tempurl GET 6000 /v1/AUTH_bf5e63572f7a420a83fcf0aa8c72c2c7\
/firstcontainer/clean.sh b3968d0207b54ece87cccc06515a89d4
/v1/AUTH_/firstcontainer/clean.sh?temp_url_sig=\
9218fc288cc09e5edd857b6a3d43cf2122b906dc&temp_url_expires=1472203614✨ The swiftclient.SwiftService API ✨
This is the higher-level API, and it’s literally designed so devs can do multiple operations asynchronously with a configurable thread pool. Slay. Documentation for each service method call can be found here: swiftclient.service.
🔐 Authentication 🔐
Okay so this section covers the various options for authenticating with a swift object store. The combinations of options required for each authentication version are detailed below. Once again, these are just a subset of those that can be used to successfully authenticate, but they are the most common and recommended.
The relevant authentication options are presented as python dictionaries that should be added to any other options you are supplying to your SwiftService instance. As indicated in the python code, you can also set these options as environment variables that will be loaded automatically if the relevant option is not specified.
The SwiftService authentication attempts to automatically select the auth version based on the combination of options specified, but supplying options from multiple different auth versions can cause unexpected behaviour.
NOTE: Leftover environment variables are a common source of confusion when authorization fails.
🔑 Keystone V3 🔑
{
...
"auth_version": environ.get('ST_AUTH_VERSION'), # Should be '3'
"os_username": environ.get('OS_USERNAME'),
"os_password": environ.get('OS_PASSWORD'),
"os_project_name": environ.get('OS_PROJECT_NAME'),
"os_project_domain_name": environ.get('OS_PROJECT_DOMAIN_NAME'),
"os_auth_url": environ.get('OS_AUTH_URL'),
...
}{
...
"auth_version": environ.get('ST_AUTH_VERSION'), # Should be '3'
"os_username": environ.get('OS_USERNAME'),
"os_password": environ.get('OS_PASSWORD'),
"os_project_id": environ.get('OS_PROJECT_ID'),
"os_project_domain_id": environ.get('OS_PROJECT_DOMAIN_ID'),
"os_auth_url": environ.get('OS_AUTH_URL'),
...
}🔑 Keystone V2 🔑
{
...
"auth_version": environ.get('ST_AUTH_VERSION'), # Should be '2.0'
"os_username": environ.get('OS_USERNAME'),
"os_password": environ.get('OS_PASSWORD'),
"os_tenant_name": environ.get('OS_TENANT_NAME'),
"os_auth_url": environ.get('OS_AUTH_URL'),
...
}🔑 Legacy Auth 🔑
{
...
"auth_version": environ.get('ST_AUTH_VERSION'), # Should be '1.0'
"auth": environ.get('ST_AUTH'),
"user": environ.get('ST_USER'),
"key": environ.get('ST_KEY'),
...
}🌸 Configuration 🌸
When you create an instance of a SwiftService, you can override a collection of default options to suit your use case. Typically, the defaults are sensible to get us started, but depending on your needs you might want to tweak them to improve performance (options affecting large objects and thread counts can significantly alter performance in the right situation).
Service level defaults and some extra options can also be overridden on a per-operation (or even in some cases per-object) basis, and you will call out which options affect which operations later in the document.
The configuration of the service API is performed using an options dictionary passed to the SwiftService during initialisation. The options available in this dictionary are described below, along with their defaults:
🎀 Options 🎀
retries=5— The number of times that the library should attempt to retry HTTP actions before giving up and reporting a failure.container_threads=10object_dd_threads=10object_uu_threads=10segment_threads=10
The above options determine the size of the available thread pools for performing swift operations. Container operations (such as listing a container) operate in the container threads, and a similar pattern applies to object and segment threads.
NOTE: Object threads are separated into two separate thread pools:uuanddd. This stands for "upload/update" and "download/delete", and the corresponding actions will be run on separate threads pools.
segment_size=None— If specified, this option enables uploading of large objects. Should the object being uploaded be larger than 5G in size, this option is mandatory otherwise the upload will fail. This option should be specified as a size in bytes.use_slo=False— Used in combination with the above option,use_slowill upload large objects as static rather than dynamic. Only static large objects provide error checking for the downloaded object, so we recommend this option.segment_container=None— Allows the user to select the container into which large object segments will be uploaded. We do not recommend changing this value as it could make locating orphaned segments more difficult in the case of errors.leave_segments=False— Setting this option to true means that when deleting or overwriting a large object, its segments will be left in the object store and must be cleaned up manually. This option can be useful when sharing large object segments between multiple objects in more advanced scenarios, but must be treated with care, as it could lead to ever increasing storage usage.changed=None— This option affects uploads and simply means that those objects which already exist in the object store will not be overwritten if the mtime and size of the source is the same as the existing object.skip_identical=False— A slightly more thorough case of the above, but rather than mtime and size uses an object's MD5 sum.yes_all=False— This option affects only download and delete, and in each case must be specified in order to download/delete the entire contents of an account. This option has no effect on any other calls.no_download=False— This option only affects download and means that all operations proceed as normal with the exception that no data is written to disk.header=[]— Used with upload and post operations to set headers on objects. Headers are specified as colon separated strings, e.g."content-type:text/plain".meta=[]— Used to set metadata on an object similarly to headers.
NOTE: Setting metadata is a destructive operation, so when updating one of many metadata values all desired metadata for an object must be re-applied.
long=False— Affects only list operations, and results in more metrics being made available in the results at the expense of lower performance.fail_fast=False— Applies to delete and upload operations, and attempts to abort queued tasks in the event of errors.prefix=None— Affects list operations; only objects with the given prefix will be returned/affected. It is not advisable to set at the service level, as those operations that call list to discover objects on which they should operate will also be affected.delimiter=None— Affects list operations, and means that listings only contain results up to the first instance of the delimiter in the object name. This is useful for working with objects containing/in their names to simulate folder structures.dir_marker=False— Affects uploads, and allows empty 'pseudofolder' objects to be created when the source of an upload isNone.checksum=True— Affects uploads and downloads. If set check md5 sum for the transfer.shuffle=False— When downloading objects, the default behaviour of the CLI is to shuffle lists of objects in order to spread the load on storage drives when multiple clients are downloading the same files to multiple locations (e.g. in the event of distributing an update). When using theSwiftServicedirectly, object downloads are scheduled in the same order as they appear in the container listing. When combined with a single download thread this means that objects are downloaded in lexically-sorted order. Setting this option toTruegives the same shuffling behaviour as the CLI.destination=None— When copying objects, this specifies the destination where the object will be copied to. The default ofNonemeans copy will be the same as source.fresh_metadata=None— When copying objects, this specifies that the object metadata on the source will not be applied to the destination object - the destination object will have a new fresh set of metadata that includes only the metadata specified in themetaoption if any at all.
Other available options can be found in swiftclient/service.py in the source code for python-swiftclient. Each SwiftService method also allows for an optional dictionary to override those specified at init time, and the appropriate docstrings show which options modify each method's behaviour.
🌈 Available Operations 🌈
Each operation provided by the service API may raise a SwiftError or ClientException for any call that fails completely (or a call which performs only one operation at an account or container level). In the case of a successful call an operation returns one of the following:
- a dictionary that's giving the results of literally one single operation 🎀
- an iterator that keeps producing result dictionaries (this is the one for calls that do multiple sub-operations, iykyk)
So a result dictionary tells you whether an individual operation ate or flopped (that's the success key, bestie), and then it's either gonna hold the successful result or an error key spelling out the error that got in the way (usually an instance of Exception).
Here's what a result dictionary is giving, fr fr:
result = {
'action': 'download_object',
'success': True,
'container': container,
'object': obj,
'path': path,
'start_time': start_time,
'finish_time': finish_time,
'headers_receipt': headers_receipt,
'auth_end_time': conn.auth_end_time,
'read_length': bytes_read,
'attempts': conn.attempts
}And every single possible action value? Right here, bestie, all of them 👇
[
'stat_account',
'stat_container',
'stat_object',
'post_account',
'post_container',
'post_object',
'list_part', # list yields zero or more 'list_part' results
'download_object',
'create_container', # from upload
'create_dir_marker', # from upload
'upload_object',
'upload_segment',
'delete_container',
'delete_object',
'delete_segment', # from delete_object operations
'capabilities',
]✨ Stat ✨
Okay so Stat is the one you call against an account, a container, or a whole list of objects to get account stats, container stats, or the tea on those objects. First two cases? You get one single dictionary back with the results of the operation. But if you hand it a list of object names, you get an iterator over the results generated for each object, because apparently one dictionary wasn't enough 💅.
The info you get back includes how much data the given object/container/account is using, plus any headers or metadata that's been set (and yes, that covers user set data and content-type and modification times — the whole package).
Check swiftclient.service.SwiftService.stat for docs generated from the method docstring, no cap.
The calls that are actually valid for this method:
stat([options])— returns stats for the configured account. Periodt.stat(<container>, [options])— returns stats for the given container.stat(<container>, <object_list>, [options])— returns stats for each of the given objects in the given container (through the returned iterator, obviously).
Results from stat are dictionaries telling you if each operation succeeded or failed. If a stat against an account or container goes off without a hitch, the method returns immediately with one of these icons:
{
'action': 'stat_account',
'success': True,
'items': items,
'headers': headers
}{
'action': 'stat_container',
'container': <container>,
'success': True,
'items': items,
'headers': headers
}And when stat is called against a list of objects? Then it returns a generator that hands you the results of individual object stat operations as they get performed on the thread pool 🧚:
{
'action': 'stat_object',
'object': <object_name>,
'container': <container>,
'success': True,
'items': items,
'headers': headers
}But if it flops 😭 the dictionary you get back will tell you the operation was not successful, and it'll include these keys:
{
'action': <'stat_object'|'stat_container'|'stat_account'>,
'object': <'object_name'>, # Only for stat with objects list
'container': <container>, # Only for stat with objects list or container
'success': False,
'error': <error>,
'traceback': <trace>,
'error_timestamp': <timestamp>
}Example
This code below shows off stat retrieving the headers for a given list of objects in a container using 20 threads. It builds a mapping from object name to headers and then pretty prints it to the log — chef's kiss 🎀
import logging
import pprint
from swiftclient.service import SwiftService
from sys import argv
logging.basicConfig(level=logging.ERROR)
logging.getLogger("requests").setLevel(logging.CRITICAL)
logging.getLogger("swiftclient").setLevel(logging.CRITICAL)
logger = logging.getLogger(__name__)
_opts = {'object_dd_threads': 20}
with SwiftService(options=_opts) as swift:
container = argv[1]
objects = argv[2:]
header_data = {}
stats_it = swift.stat(container=container, objects=objects)
for stat_res in stats_it:
if stat_res['success']:
header_data[stat_res['object']] = stat_res['headers']
else:
logger.error(
'Failed to retrieve stats for %s' % stat_res['object']
)
pprint.pprint(header_data)🌸 List 🌸
List can be called against an account or a container to pull the containers or objects chilling inside them. Every call returns an iterator that hands you pages of results (and by default that's up to 10000 results in each page, bestie — it's giving efficiency 💅).
See swiftclient.service.SwiftService.list for docs generated from the method docstring.
If the container or account you asked for doesn't even exist, the list method will raise a SwiftError. But for every other success/failure situation, you get a dictionary back. Each successfully listed page returns a dictionary like this:
{
'action': <'list_account_part'|'list_container_part'>,
'container': <container>, # Only for listing a container
'prefix': <prefix>, # The prefix of returned objects/containers
'success': True,
'listing': [Item], # A list of results
# (only in the event of success)
'marker': <marker> # The last item name in the list
# (only in the event of success)
}And an item is holding these keys right here:
{
'name': <name>,
'bytes': 10485760,
'last_modified': '2014-12-11T12:02:38.774540',
'hash': 'fb938269cbeabe4c234e1127bbd3b74a',
'content_type': 'application/octet-stream',
'meta': <metadata> # Full metadata listing from stat'ing each object
# this key only exists if 'long' is specified in options
}Now if listing an account or container that does exist fails, you get a failure dictionary described like this:
{
'action': <'list_account_part'|'list_container_part'>,,
'container': container, # Only for listing a container
'prefix': options['prefix'],
'success': success,
'marker': marker,
'error': error,
'traceback': <trace>,
'error_timestamp': <timestamp>
}Example
The code below demonstrates the use of list to list all items in a container that are over 10MiB in size 🔥:
import logging
from swiftclient.service import SwiftService, SwiftError
from sys import argv
logging.basicConfig(level=logging.ERROR)
logging.getLogger("requests").setLevel(logging.CRITICAL)
logging.getLogger("swiftclient").setLevel(logging.CRITICAL)
logger = logging.getLogger(__name__)
container = argv[1]
minimum_size = 10*1024**2
with SwiftService() as swift:
try:
list_parts_gen = swift.list(container=container)
for page in list_parts_gen:
if page["success"]:
for item in page["listing"]:
i_size = int(item["bytes"])
if i_size > minimum_size:
i_name = item["name"]
i_etag = item["hash"]
print(
"%s [size: %s] [etag: %s]" %
(i_name, i_size, i_etag)
)
else:
raise page["error"]
except SwiftError as e:
logger.error(e.value)🦋 Post 🦋
Post can be called against an account, a container, or a list of objects in order to update the metadata attached to the given items. First two cases? A single dictionary comes back with the results of the operation. But when a list of objects is supplied, you get an iterator over the results generated for each object post — one per object, no skipping 💅.
Each element of the object list can be either a plain string of the object name, or a SwiftPostObject which allows finer control over the options and metadata applied to each of the individual post operations. When it's just a string for the object name, the options and metadata applied are a combination of those supplied to the call to post() and the defaults of the SwiftService object.
If the given container or account doesn't exist, the post method will raise a SwiftError. But successful metadata update results are dictionaries like this:
{
'action': <'post_account'|'post_container'|'post_object'>,
'success': True,
'container': <container>,
'object': <object>,
'headers': {},
'response_dict': <HTTP response details>
}NOTE: Updating user metadata keys will not only add any specified keys, but will also remove user metadata that has previously been set. This means that each time user metadata is updated, the complete set of desired key-value pairs must be specified. Like, the whole set — every time. Don't come crying to me when your old keys vanish 😭
Example
The code below demonstrates the use of post to set an archive folder in a given container to expire after a 24 hour delay:
import logging
from swiftclient.service import SwiftService, SwiftError
from sys import argv
logging.basicConfig(level=logging.ERROR)
logging.getLogger("requests").setLevel(logging.CRITICAL)
logging.getLogger("swiftclient").setLevel(logging.CRITICAL)
logger = logging.getLogger(__name__) container = argv[1]
with SwiftService() as swift:
try:
list_options = {"prefix": "archive_2016-01-01/"}
list_parts_gen = swift.list(container=container)
for page in list_parts_gen:
if page["success"]:
objects = [obj["name"] for obj in page["listing"]]
post_options = {"header": "X-Delete-After:86400"}
for post_res in swift.post(
container=container,
objects=objects,
options=post_options):
if post_res['success']:
print("Object '%s' POST success" % post_res['object'])
else:
print("Object '%s' POST failed" % post_res['object'])
else:
raise page["error"]
except SwiftError as e:
logger.error(e.value)🦋 Download 🦋
Okay so download is the one that pulls your stuff back down to you, and she's flexible like that — you can aim her at a whole account, at a single container, or at a list of objects inside one given container. Every element of that object list is just a plain string that spells out the full name of the object you want 💖.
Want the entire account's contents? Then you gotta set the value of yes_all to True, and you put that inside the options dictionary that you hand to either the SwiftService instance or to the call to download itself. Both doors work, bestie 🫶.
If the container or account you named doesn't exist, download will raise a SwiftError — no cap, that's the deal. Otherwise you get back an iterator over the results generated for each object download ✨.
See swiftclient.service.SwiftService.download for docs generated from the method docstring 📖
For each object that downloads successfully, the results the iterator hands you are a dictionary that looks exactly like this (heads up though: results are not returned for completed container or object segment downloads 🙈):
{
'action': 'download_object',
'container': <container>,
'object': <object name>,
'success': True,
'path': <local path to downloaded object>,
'pseudodir': <if true, the download created an empty directory>,
'start_time': <time download started>,
'end_time': <time download completed>,
'headers_receipt': <time the headers from the object were retrieved>,
'auth_end_time': <time authentication completed>,
'read_length': <bytes_read>,
'attempts': <attempt count>,
'response_dict': <HTTP response details>
}And any failure comes back as a failure dictionary like this one (the original page literally says "uploading" in this line — it's giving copy-paste energy 💀 — but this is the shape you get when a download flops):
{
'action': 'download_object',
'container': <container>,
'object': <object name>,
'success': False,
'path': <local path of the failed download>,
'pseudodir': <if true, the failed download was an empty directory>,
'attempts': <attempt count>,
'error': <error>,
'traceback': <trace>,
'error_timestamp': <timestamp>,
'response_dict': <HTTP response details>
}🌸 Example 🌸
So this next code block is literally the demo of using download to grab all the PNG images out of a dated archive folder in a given container. Iconic. Filtering queen behavior 👑:
import logging
from swiftclient.service import SwiftService, SwiftError
from sys import argv
logging.basicConfig(level=logging.ERROR)
logging.getLogger("requests").setLevel(logging.CRITICAL)
logging.getLogger("swiftclient").setLevel(logging.CRITICAL)
logger = logging.getLogger(__name__)
def is_png(obj):
return (
obj["name"].lower().endswith('.png') or
obj["content_type"] == 'image/png'
)
container = argv[1]
with SwiftService() as swift:
try:
list_options = {"prefix": "archive_2016-01-01/"}
list_parts_gen = swift.list(container=container)
for page in list_parts_gen:
if page["success"]:
objects = [
obj["name"] for obj in page["listing"] if is_png(obj)
]
for down_res in swift.download(
container=container,
objects=objects):
if down_res['success']:
print("'%s' downloaded" % down_res['object'])
else:
print("'%s' download failed" % down_res['object'])
else:
raise page["error"]
except SwiftError as e:
logger.error(e.value)🔥 Upload 🔥
Upload is always called against an account and a container, and always with a list of objects to upload. Periodt 💅.
Each element of that object list can be either:
- a plain string that details the path of the object to upload, or
- a
SwiftUploadObject, which is the fancy one that lets you have finer control over some aspects of the individual operations ✨.
When you just hand over a simple string to specify a file to upload, then the name of the object that gets uploaded is the full path of that specified file, and the options used for the upload are the ones you supplied to the call to upload. Simple, clean, no drama 🌈.
But constructing a SwiftUploadObject? That's how you get to supply an object name for the uploaded file, and modify the options used by upload at the granularity of individual files 👑. Per-file main character energy, we love to see it.
If the given container or account doesn't exist, upload will raise a SwiftError. Otherwise you get an iterator over the results generated for each object upload 🎀.
See swiftclient.service.SwiftService.upload for docs generated from the method docstring 📖
For each successfully uploaded object (or object segment, bestie — segments count too), the results the iterator gives you are dictionaries like these:
{
'action': 'upload_object',
'container': <container>,
'object': <object name>,
'success': True,
'status': <'uploaded'|'skipped-identical'|'skipped-changed'>,
'attempts': <attempt count>,
'response_dict': <HTTP response details>
} {
'action': 'upload_segment',
'for_container': <container>,
'for_object': <object name>,
'segment_index': <segment_index>,
'segment_size': <segment_size>,
'segment_location': <segment_path>
'segment_etag': <etag>,
'log_line': <object segment n>
'success': True,
'response_dict': <HTTP response details>,
'attempts': <attempt count>
}And any failure uploading an object returns a failure dictionary as described below 😭:
{
'action': 'upload_object',
'container': <container>,
'object': <object name>,
'success': False,
'attempts': <attempt count>,
'error': <error>,
'traceback': <trace>,
'error_timestamp': <timestamp>,
'response_dict': <HTTP response details>
} {
'action': 'upload_segment',
'for_container': <container>,
'for_object': <object name>,
'segment_index': <segment_index>,
'segment_size': <segment_size>,
'segment_location': <segment_path>,
'log_line': <object segment n>,
'success': False,
'error': <error>,
'traceback': <trace>,
'error_timestamp': <timestamp>,
'response_dict': <HTTP response details>,
'attempts': <attempt count>
}💜 Example 💜
Okay this one's a banger: it demos upload uploading all the files and folders in a given directory, and renaming each object by replacing the root directory name with 'my-<d>-objects', where <d> is the name of the uploaded directory. Rename slay fr fr ✨:
import logging
from os import walk
from os.path import join
from swiftclient.multithreading import OutputManager
from swiftclient.service import SwiftError, SwiftService, SwiftUploadObject
from sys import argv
logging.basicConfig(level=logging.ERROR)
logging.getLogger("requests").setLevel(logging.CRITICAL)
logging.getLogger("swiftclient").setLevel(logging.CRITICAL)
logger = logging.getLogger(__name__)
_opts = {'object_uu_threads': 20}
dir = argv[1]
container = argv[2]
with SwiftService(options=_opts) as swift, OutputManager() as out_manager:
try:
# Collect all the files and folders in the given directory
objs = []
dir_markers = []
for (_dir, _ds, _fs) in walk(dir):
if not (_ds + _fs):
dir_markers.append(_dir)
else:
objs.extend([join(_dir, _f) for _f in _fs])
# Now that we've collected all the required files and dir markers
# build the ``SwiftUploadObject``s for the call to upload
objs = [
SwiftUploadObject(
o, object_name=o.replace(
dir, 'my-%s-objects' % dir, 1
)
) for o in objs
]
dir_markers = [
SwiftUploadObject(
None, object_name=d.replace(
dir, 'my-%s-objects' % dir, 1
), options={'dir_marker': True}
) for d in dir_markers
]
# Schedule uploads on the SwiftService thread pool and iterate
# over the results
for r in swift.upload(container, objs + dir_markers):
if r['success']:
if 'object' in r:
print(r['object'])
elif 'for_object' in r:
print(
'%s segment %s' % (r['for_object'],
r['segment_index'])
)
else:
error = r['error']
if r['action'] == "create_container":
logger.warning(
'Warning: failed to create container '
"'%s'%s", container, error
)
elif r['action'] == "upload_object":
logger.error(
"Failed to upload object %s to container %s: %s" %
(container, r['object'], error)
)
else:
logger.error("%s" % error)
except SwiftError as e:
logger.error(e.value)🗑️ Delete
Okay bestie, so Delete — you can call it against an account or a container and it will remove the containers or objects chilling inside them. Every single call to delete hands you back an iterator over the results of each resulting sub-request. Cute, right? 💅
And here's the ✨spicy✨ bit: if you're requesting a large number of delete operations and your target swift cluster is running the bulk middleware, then the call to SwiftService.delete will make use of bulk operations. When that happens, the returned result iterator gives you bulk_delete results instead of the individual delete_object, delete_container or delete_segment results. iykyk 😌
If you want the full tea, see swiftclient.service.SwiftService.delete for docs generated from the method docstring.
For each successfully deleted container, object or segment, the results returned by the iterator will be a dictionary exactly like one of these bad boys:
{
'action': <'delete_object'|'delete_segment'>,
'container': <container>,
'object': <object name>,
'success': True,
'attempts': <attempt count>,
'response_dict': <HTTP response details>
}{
'action': 'delete_container',
'container': <container>,
'success': True,
'response_dict': <HTTP response details>,
'attempts': <attempt count>
}{
'action': 'bulk_delete',
'container': <container>,
'objects': <[objects]>,
'success': True,
'attempts': <attempt count>,
'response_dict': <HTTP response details>
}Any failure in a delete operation — and yeah, it happens, no cap 😭 — will return a failure dictionary as described below:
{
'action': ('delete_object'|'delete_segment'),
'container': <container>,
'object': <object name>,
'success': False,
'attempts': <attempt count>,
'error': <error>,
'traceback': <trace>,
'error_timestamp': <timestamp>,
'response_dict': <HTTP response details>
}{
'action': 'delete_container',
'container': <container>,
'success': False,
'error': <error>,
'traceback': <trace>,
'error_timestamp': <timestamp>,
'response_dict': <HTTP response details>,
'attempts': <attempt count>
}{
'action': 'bulk_delete',
'container': <container>,
'objects': <[objects]>,
'success': False,
'attempts': <attempt count>,
'error': <error>,
'traceback': <trace>,
'error_timestamp': <timestamp>,
'response_dict': <HTTP response details>
}Example
The code below is literally a whole demo of using delete to remove a given list of objects from a specified container. As the objects get yeeted, the transaction ID of the relevant request is printed along with the object name and the number of attempts required. And why do we print the transaction ID? Because then the printed operations can be easily linked to events in the swift server logs — main character detective energy 🔍✨
import logging
from swiftclient.service import SwiftService
from sys import argv
logging.basicConfig(level=logging.ERROR)
logging.getLogger("requests").setLevel(logging.CRITICAL)
logging.getLogger("swiftclient").setLevel(logging.CRITICAL)
logger = logging.getLogger(__name__)
_opts = {'object_dd_threads': 20}
container = argv[1]
objects = argv[2:]
with SwiftService(options=_opts) as swift:
del_iter = swift.delete(container=container, objects=objects)
for del_res in del_iter:
c = del_res.get('container', '')
o = del_res.get('object', '')
a = del_res.get('attempts')
if del_res['success'] and not del_res['action'] == 'bulk_delete':
rd = del_res.get('response_dict')
if rd is not None:
t = dict(rd.get('headers', {}))
if t:
print(
'Successfully deleted {0}/{1} in {2} attempts '
'(transaction id: {3})'.format(c, o, a, t)
)
else:
print(
'Successfully deleted {0}/{1} in {2} '
'attempts'.format(c, o, a)
)📋 Copy
Copy — you call it to copy an object or to update the metadata on the given items. Two-for-one, we love to see it 🫶
Each element of the object list may be one of these two things:
- a plain string of the object name, OR
- a
SwiftCopyObject— which gives you finer control over the options applied to each of the individual copy operations (destination,fresh_metadata,options).
Let's talk Destination: it should be in the format /container/object. If it's not set, the object gets copied onto itself (glow-up in place, honestly iconic). And Fresh_metadata? That sets the mode of operation on metadata: if it's not set, the current object user metadata will be copied/preserved; if it is set, all current user metadata will be removed. Poof. Gone. 💀
It returns an iterator over the results generated for each object copy — and yes, that may also include the results of creating destination containers.
When a string is given for the object name, destination and fresh metadata will default to None and None, which result in adding metadata to existing objects.
Successful copy results are dictionaries exactly as described below:
{
'action': 'copy_object',
'success': True,
'container': <container>,
'object': <object>,
'destination': <destination>,
'headers': {},
'fresh_metadata': <boolean>,
'response_dict': <HTTP response details>
}Any failure in a copy operation will return a failure dictionary as described below:
{
'action': 'copy_object',
'success': False,
'container': <container>,
'object': <object>,
'destination': <destination>,
'headers': {},
'fresh_metadata': <boolean>,
'response_dict': <HTTP response details>,
'error': <error>,
'traceback': <traceback>,
'error_timestamp': <timestamp>
}Example
The code below demonstrates the use of copy to add new user metadata for objects a and b, and to copy object c to d (with added metadata). Slay ✨
import logging
from swiftclient.service import SwiftService, SwiftCopyObject, SwiftError
logging.basicConfig(level=logging.ERROR)
logging.getLogger("requests").setLevel(logging.CRITICAL)
logging.getLogger("swiftclient").setLevel(logging.CRITICAL)
logger = logging.getLogger(__name__)
with SwiftService() as swift:
try:
obj = SwiftCopyObject("c", {"destination": "/cont/d"})
for i in swift.copy(
"cont", ["a", "b", obj],
{"meta": ["foo:bar"], "destination": "/cc"}):
if i["success"]:
if i["action"] == "copy_object":
print(
"object %s copied from /%s/%s" %
(i["destination"], i["container"], i["object"])
)
elif i["action"] == "create_container":
print(
"container %s created" % i["container"]
)
else:
if "error" in i and isinstance(i["error"], Exception):
raise i["error"]
except SwiftError as e:
logger.error(e.value)🔮 Capabilities
Capabilities can be called against an account or a particular proxy URL in order to determine the capabilities of the swift cluster. These capabilities include details about configuration options and the middlewares that are installed in the proxy pipeline. It's giving "tell me everything about yourself" first-date energy 🙈
See swiftclient.service.SwiftService.capabilities for docs generated from the method docstring.
For each successful call to list capabilities, a result dictionary will be returned with the contents described below:
{
'action': 'capabilities',
'timestamp': <time of the call>,
'success': True,
'capabilities': <dictionary containing capability details>
}The contents of the capabilities dictionary contain the core swift capabilities under the key swift; all other keys show the configuration options for additional middlewares deployed in the proxy pipeline. An example capabilities dictionary is given below — look at all these keys, honestly obsessed 😍:
{
'account_quotas': {},
'bulk_delete': {
'max_deletes_per_request': 10000,
'max_failed_deletes': 1000
},
'bulk_upload': {
'max_containers_per_extraction': 10000,
'max_failed_extractions': 1000
},
'container_quotas': {},
'container_sync': {'realms': {}},
'formpost': {},
'keystoneauth': {},
'slo': {
'max_manifest_segments': 1000,
'max_manifest_size': 2097152,
'min_segment_size': 1048576
},
'swift': {
'account_autocreate': True,
'account_listing_limit': 10000,
'allow_account_management': True,
'container_listing_limit': 10000,
'extra_header_count': 0,
'max_account_name_length': 256,
'max_container_name_length': 256,
'max_file_size': 5368709122,
'max_header_size': 8192,
'max_meta_count': 90,
'max_meta_name_length': 128,
'max_meta_overall_size': 4096,
'max_meta_value_length': 256,
'max_object_name_length': 1024,
'policies': [
{'default': True, 'name': 'Policy-0'}
],
'strict_cors_mode': False,
'version': '2.2.2'
},
'tempurl': {
'methods': ['GET', 'HEAD', 'PUT']
}
}Example
The code below demonstrates the use of capabilities to determine if the Swift cluster supports static large objects, and if so, the maximum number of segments that can be described in a single manifest file, along with the size restrictions on those objects:
import logging
from swiftclient.exceptions import ClientException
from swiftclient.service import SwiftService
logging.basicConfig(level=logging.ERROR)
logging.getLogger("requests").setLevel(logging.CRITICAL)
logging.getLogger("swiftclient").setLevel(logging.CRITICAL)
logger = logging.getLogger(__name__)with SwiftService() as swift:
try:
capabilities_result = swift.capabilities()
capabilities = capabilities_result['capabilities']
if 'slo' in capabilities:
print('SLO is supported')
else:
print('SLO is not supported')
except ClientException as e:
logger.error(e.value)The swiftclient.Connection API 🧚
okay bestie, this here is the low level API — the one that hands you methods for authentication plus methods that line up one-to-one with the individual REST API calls described over in the swift documentation. it's giving backstage pass, no cap.
for all the usage deets, go read the client docs at: swiftclient.client 💅
Authentication 🔐
so this bit covers all the various combos of kwargs you need when you're creating an instance of the Connection object so it can talk to a swift object store. the combos of options needed for each auth version are listed right below, but heads up — these are just a subset of everything that could technically authenticate you. they're just the most common and the recommended ones. slay.
Keystone Session 🌸
from keystoneauth1 import session
from keystoneauth1.identity import v3
# Create a password auth plugin
auth = v3.Password(auth_url='http://127.0.0.1:5000/v3/',
username='tester',
password='testing',
user_domain_name='Default',
project_name='Default',
project_domain_name='Default')
# Create session
keystone_session = session.Session(auth=auth)
# Create swiftclient Connection
swift_conn = Connection(session=keystone_session)Keystone v3 💜
_authurl = 'http://127.0.0.1:5000/v3/'
_auth_version = '3'
_user = 'tester'
_key = 'testing'
_os_options = {
'user_domain_name': 'Default',
'project_domain_name': 'Default',
'project_name': 'Default'
}
conn = Connection(
authurl=_authurl,
user=_user,
key=_key,
os_options=_os_options,
auth_version=_auth_version
)Keystone v2 🦋
_authurl = 'http://127.0.0.1:5000/v2.0/'
_auth_version = '2'
_user = 'tester'
_key = 'testing'
_tenant_name = 'test'
conn = Connection(
authurl=_authurl,
user=_user,
key=_key,
tenant_name=_tenant_name,
auth_version=_auth_version
)Legacy Auth 🕰️
_authurl = 'http://127.0.0.1:8080/'
_auth_version = '1'
_user = 'tester'
_key = 'testing'
_tenant_name = 'test'
conn = Connection(
authurl=_authurl,
user=_user,
key=_key,
tenant_name=_tenant_name,
auth_version=_auth_version
)Examples ✨
okay so here's where we serve up some simple little code examples that show off the Connection API in action. the full details of every option and method the Connection API gives you live in the docstring-generated documentation: swiftclient.client. go off.
List the available containers: 👑
resp_headers, containers = conn.get_account()
print("Response headers: %s" % resp_headers)
for container in containers:
print(container)Create a new container: 🎀
container = 'new-container'
conn.put_container(container)
resp_headers, containers = conn.get_account()
if container in containers:
print("The container was created")Create a new object with the contents of a local text file: 📄
container = 'new-container'
with open('local.txt', 'r') as local:
conn.put_object(
container,
'local_object.txt',
contents=local,
content_type='text/plain'
)Confirm presence of the object: 🔍
obj = 'local_object.txt'
container = 'new-container'
try:
resp_headers = conn.head_object(container, obj)
print('The object was successfully created')
except ClientException as e:
if e.http_status = '404':
print('The object was not found')
else:
print('An error occurred checking for the existence of the object')Download the created object: 💾
obj = 'local_object.txt'
container = 'new-container'
resp_headers, obj_contents = conn.get_object(container, obj)
with open('local_copy.txt', 'w') as local:
local.write(obj_contents)Delete the created object: 💀
obj = 'local_object.txt'
container = 'new-container'
try:
conn.delete_object(container, obj)
print("Successfully deleted the object")
except ClientException as e:
print("Failed to delete the object with error: %s" % e)CODE-GENERATED DOCUMENTATION 🤖
no cap, this whole chunk got spat out by the code itself. it's the raw reference, bestie — read it slow.
swiftclient 🐍
OpenStack Swift Python client binding. iconic, honestly.
swiftclient.authv1 🔑
Authentication plugin for keystoneauth to support v1 endpoints.
way back in the long-long ago, there was no Keystone at all. Swift used an auth mechanism that we now call "v1", and it used only HTTP headers. auth requests and responses looked a lil something like this:
> GET /auth/v1.0 HTTP/1.1
> Host: <swift server>
> X-Auth-User: <tenant>:<user>
> X-Auth-Key: <password>
>
< HTTP/1.1 200 OK
< X-Storage-Url: http://<swift server>/v1/<tenant account>
< X-Auth-Token: <token>
< X-Storage-Token: <token>
<this plugin is your bridge 🌉 — it lets Keystone sessions (and the clients that use them, like python-openstackclient) talk to those old auth endpoints that are still rocking this mechanism. think tempauth, swauth, or https://identity.api.rackspacecloud.com/v1.0. vintage but still valid, iykyk.
class swiftclient.authv1.AccessInfoV1(auth_url, storage_url, account, username, auth_token, token_life)
An object for encapsulating a raw v1 auth token. lil container for your token, basically 🫙
classmethod from_state(data)— Deserialize the given state.- Returns: a new
AccessInfoV1object with the given state
- Returns: a new
get_state()— Serialize the current state.will_expire_soon(stale_duration)— Determines if expiration is about to occur.- Returns: true if expiration is within the given duration ⏳
class swiftclient.authv1.PasswordLoader
Option handling for the v1password plugin. it's the loader, it loads. periodt.
property available: bool— Return if the plugin is available for loading.
if a plugin is missing dependencies, or for some other reason should not be available to the current system, it should override this property and return False to exclude itself from the plugin list.
Return type: bool
create_plugin(**kwargs: Any) -> T— Create a plugin from the options available for the loader.
given the options that were specified by the loader, create an appropriate plugin. you can override this function in your loader.
this used to be specified by providing the plugin_class property and this is still supported, however specifying a property didn't let you choose a plugin type based upon the options that were presented.
override this function if you wish to return different plugins based on the options presented, otherwise you can simply provide the plugin_class property.
Added 2.9 🆕
get_options()— Return the list of parameters associated with the auth plugin.
this list may be used to generate CLI or config arguments.
load_from_options(**kwargs: Any) -> T— Create a plugin from the arguments retrieved fromget_options.
a client can override this function to do argument validation or to handle differences between the registered options and what is required to create the plugin.
load_from_options_getter(getter: Callable[[opts.Opt], Any], **kwargs: Any) -> T— Load a plugin from getter function that returns appropriate values.
to handle cases other than the provided CONF and CLI loading you can specify a custom loader function that will be queried for the option value. the getter is a function that takes a keystoneauth1.loading.Opt and returns a value to load with.
Parameters: getter (callable) — A function that returns a value for the given opt.
Returns: An authentication Plugin.
Return type: keystoneauth1.plugin.BaseAuthPlugin
plugin_class— alias ofPasswordPlugin
class swiftclient.authv1.PasswordPlugin(auth_url, username, password, project_name=None, reauthenticate=True)
A plugin for authenticating with a username and password. classic combo, still ate 💅
subclassing from BaseIdentityPlugin gets us a few niceties, like handling token invalidation and locking during authentication. free perks, we love that.
Parameters:
auth_url(string) — Identity v1 endpoint for authorization.username(string) — Username for authentication.password(string) — Password for authentication.project_name(string) — Swift account to use after authentication. we use'project_name'to be consistent with other auth plugins.reauthenticate(string) — Whether to allow re-authentication.
access_class— alias ofAccessInfoV1get_access(session: Session, **kwargs: Any) -> AccessInfo— Fetch or return a currentAccessInfoobject.
if a valid AccessInfo is present then it is returned, otherwise a new one will be fetched.
Parameters: session (keystoneauth1.session.Session) — A session object that can be used for communication.
Raises: keystoneauth1.exceptions.http.HttpError — An error from an invalid HTTP response.
Returns: Valid AccessInfo
Return type: keystoneauth1.access.AccessInfo 🌈
get_all_version_data(session: Session, interface: str | list[str] | None = 'public', region_name: str | None = None, service_type: str | None = None) -> dict[str, dict[str, dict[str, list[VersionData]]]]
Okay so this one pulls version data for every service in the catalog 💅 — all of them, at once, no cap.
Parameters
session(keystoneauth1.session.Session) — a session object you can use for communication 🫶interface— the type of endpoint you want version data for. Can be a single value or a list of values. A value ofNonemeans "query all the interfaces, idc". (optional, defaults topublic)region_name(string) — the region of the endpoints you want version data for. A value ofNonemeans all regions get queried. (optional, defaults toNone)service_type(string) — limits the version data to a single service. (optional, defaults toNone)
Returns
A dictionary keyed by region_name 💖 — and its values hold dictionaries keyed by interface, whose values are a list of VersionData. Nested. Very nested. Slay.
get_api_major_version(session: Session, *, endpoint_override: str | None = None, service_type: str | None = None, interface: str | None = None, region_name: str | None = None, service_name: str | None = None, version: str | None = None, allow: dict[str, Any] | None = None, allow_version_hack: bool = True, skip_discovery: bool = False, discover_versions: bool = False, min_version: str | int | float | Iterable[str | int | float] | None = None, max_version: str | int | float | Iterable[str | int | float] | None = None, **kwargs: Any) -> tuple[int | float, ...] | None
Returns the major API version for a service ✨
If there's no valid token kicking around, a fresh one gets fetched using the session and kwargs 🔥
version, min_version and max_version can all be given either as a string or as a tuple — your call, bestie.
Valid interface types: public or publicURL, internal or internalURL, admin or adminURL 👑
Parameters
session(keystoneauth1.session.Session) — a session object you can use for communication.endpoint_override(str) — the URL to use for version discovery.service_type(string) — the type of service to look up the endpoint for. This plugin will returnNone(i.e. failure 💀) ifservice_typeisn't provided.interface— the type of endpoint. Can be a single value or a list of values. If it's a list, they get looked for in order of preference 🎀 Can also bekeystoneauth1.plugin.AUTH_INTERFACEto signal that theauth_urlshould be used instead of whatever's in the catalog. (optional, defaults topublic)region_name(string) — the region the endpoint should exist in. (optional)service_name(string) — the name of the service in the catalog. (optional)version— the minimum version number required for this endpoint. (optional)allow(dict) — extra filters to pass when discovering API versions. (optional)allow_version_hack(bool) — lets keystoneauth hack up catalog URLs to support older schemes. (optional, defaultTrue)skip_discovery(bool) — whether to skip version discovery even if a version has been given. Genuinely useful whenendpoint_overrideor similar has been given and grabbing extra info about the endpoint isn't useful 🙅♀️discover_versions(bool) — whether to get version metadata from the version discovery document even if it's not necessary to fulfill the major version request. Defaults toFalsebecauseget_endpointdoesn't need metadata. (optional, defaults toFalse)min_version— the minimum version that's acceptable. Mutually exclusive withversion. Ifmin_versionis given with nomax_version, it's as if max version is'latest'. (optional)max_version— the maximum version that's acceptable. Mutually exclusive withversion. Ifmin_versionis given with nomax_version, it's as if max version is'latest'. (optional)
Raises keystoneauth1.exceptions.http.HttpError — an error from an invalid HTTP response 😭
Returns
The major version of the API of the service discovered.
Return type
tuple or None
NOTE: Implementation notes follow. Users should not need to wrap their head around these implementation notes. get_api_major_version should do what is expected with the least possible cost while still consistently returning a value if possible. 💜
There are many cases where the major version can be satisfied without actually calling the discovery endpoint (like when the version is just in the URL). If the user has a cloud with the versioned endpoint https://volume.example.com/v3 in the catalog for the block-storage service and they do:
client = adapter.Adapter(
session,
service_type='block-storage',
min_version=2,
max_version=3,
)
volume_version = client.get_api_major_version()Then the version will actually come back with no API calls other than getting the token. For that reason, get_api_major_version() first calls get_endpoint_data() with discover_versions=False ⭐
If their catalog has an unversioned endpoint https://volume.example.com for the block-storage service and they do this:
client = adapter.Adapter(session, service_type='block-storage')then client is now set up to "use whatever is in the catalog". Since the URL doesn't have a version, get_endpoint_data() with discover_versions=False will result in api_version=None. (No version was requested, so it didn't need to do the round trip 🦋)
In order to find out what version the endpoint actually is, we must make a round trip. Therefore, if api_version is None after the first call, get_api_major_version() will make a second call to get_endpoint_data() with discover_versions=True 💫
get_auth_ref(session, **kwargs)
Obtain a token from a v1 endpoint 🔑
This function should not be called independently — it's expected to be invoked via the do_authenticate function.
It gets invoked if the AccessInfo object cached by the plugin isn't valid. So plugins should always fetch a new AccessInfo when invoked. If you're just looking to retrieve the current auth data, then you should use get_access instead, bestie 🙈
Parameters
session— a session object you can use for communication.
Returns
Token access information 💖
get_auth_state()
Retrieves the current authentication state for the plugin 🌸
Returns
raw python data (which can be JSON serialized) that can be moved into another plugin (of the same type) so it has the same authenticated state ✨
get_cache_id() -> str | None
Fetches an identifier that uniquely identifies the auth options 🎀
The returned identifier doesn't need to be decomposable or otherwise provide any way to recreate the plugin — it's just an id, not a recipe.
This string MUST change if any of the parameters that are used to uniquely identify this plugin change. It should not change upon a reauthentication of the plugin. Periodt.
Returns
A unique string for the set of options.
Return type
str, or None if this is unsupported or unavailable.
get_cache_id_elements()
Gets the elements for this auth plugin that make it unique 💅
get_connection_params(session: ks_session.Session) -> ConnectionParams
Returns any additional connection parameters required for the plugin 🌈
Parameters
session(keystoneauth1.session.Session) — the session object that theauth_pluginbelongs to.
Returns
Parameters that are passed to the requests library. Only the cert and verify parameters may be returned 👀
Return type
dict
get_discovery(*args, **kwargs)
Returns the discovery object for a URL 🦋
It checks the session and the plugin cache to see if we've already performed discovery on the URL — if so, it returns that. Otherwise it creates a new discovery object, caches it, and returns it. Iconic, honestly 💜
This function is expected to be used by subclasses and should not be needed by users. Like, it's internal machinery bestie — if you're calling it as a user you're probably off-roading 🫶
Parameters
session(keystoneauth1.session.Session) — a session object to discover with.url(str) — the url to look up.authenticated(bool) — include a token in the discovery call. (optional) Defaults toNone, which means use a token if a plugin is installed 💅
Raises
keystoneauth1.exceptions.discovery.DiscoveryFailure— if for some reason the lookup fails 😭keystoneauth1.exceptions.http.HttpError— an error from an invalid HTTP response.
Returns
A discovery object with the results of looking up that URL. ✨
🎀 get_endpoint(session, interface='public', **kwargs)
Return an endpoint for the client. That's it, that's the whole vibe 💖
🦋 get_endpoint_data(...)
get_endpoint_data(session: Session, *, endpoint_override: str |
None = None, discover_versions: bool = True, service_type: str |
None = None, interface: str | None = None, region_name: str |
None = None, service_name: str | None = None, allow: dict[str,
Any] | None = None, allow_version_hack: bool = True,
skip_discovery: bool = False, min_version: str | int | float |
Iterable[str | int | float] | None = None, max_version: str | int
| float | Iterable[str | int | float] | None = None, **kwargs:
Any) -> EndpointData | NoneReturn a valid endpoint data for a service. yasss, the big one 👑
If a valid token is not present then a new one will be fetched using the session and kwargs. No token? No problem, we go get one 💅
version, min_version and max_version can all be given either as a string or a tuple.
Valid interface types: public or publicURL, internal or internalURL, admin or 'adminURL
Parameters
session(keystoneauth1.session.Session) — a session object that can be used for communication.endpoint_override(str) — URL to use instead of looking in the catalog. Catalog lookup will be skipped, but version discovery will be run. Setsallow_version_hacktoFalse(optional).discover_versions(bool) — whether to get version metadata from the version discovery document even if it's not neccessary to fulfill the major version request. (optional, defaults toTrue)service_type(string) — the type of service to lookup the endpoint for. This plugin will returnNone(failure) ifservice_typeis not provided.interface— type of endpoint. Can be a single value or a list of values. If it's a list of values, they will be looked for in order of preference. Can also bekeystoneauth1.plugin.AUTH_INTERFACEto indicate that theauth_urlshould be used instead of the value in the catalog. (optional, defaults topublic)region_name(string) — the region the endpoint should exist in. (optional)service_name(string) — the name of the service in the catalog. (optional)allow(dict) — extra filters to pass when discovering API versions. (optional)allow_version_hack(bool) — allow keystoneauth to hack up catalog URLS to support older schemes. (optional, defaultTrue)skip_discovery(bool) — whether to skip version discovery even if a version has been given. This is useful ifendpoint_overrideor similar has been given and grabbing additional information about the endpoint is not useful. 💀min_version— the minimum version that is acceptable. Mutually exclusive withversion. Ifmin_versionis given with nomax_versionit is as if max version is'latest'. (optional)max_version— the maximum version that is acceptable. Mutually exclusive withversion. Ifmin_versionis given with nomax_versionit is as if max version is'latest'. (optional)kwargs— ignored. literally just vibing there 🙈
Raises keystoneauth1.exceptions.http.HttpError — an error from an invalid HTTP response.
Returns
Valid EndpointData or None if not available.
Return type
keystoneauth1.discover.EndpointData or None
🌸 get_headers(session: ks_session.Session) -> dict[str, str] | None
Fetch authentication headers for message. 📨
This is a more generalized replacement of the older get_token to allow plugins to specify different or additional authentication headers to the OpenStack standard 'X-Auth-Token' header.
How the authentication headers are obtained is up to the plugin. If the headers are still valid they may be re-used, retrieved from cache or the plugin may invoke an authentication request against a server. whatever the plugin feels like, honestly ✨
The default implementation of get_headers calls the get_token method to enable older style plugins to continue functioning unchanged. Subclasses should feel free to completely override this function to provide the headers that they want. main character energy fr 💅
Returning None will indicate that no token was able to be retrieved and that authorization was a failure. Adding no authentication data can be achieved by returning an empty dictionary. empty dict ≠ failure, bestie — write that down 📝
Parameters
session(keystoneauth1.session.Session) — the session object that theauth_pluginbelongs to.
Returns
Headers that are set to authenticate a message or None for failure. Note that when checking this value that the empty dict is a valid, non-failure response.
Return type
dict
🧚 get_project_id(session: Session) -> str | None
Return the project id that we are authenticated to. 🫶
Wherever possible the project id should be inferred from the token however there are certain URLs and other places that require access to the currently authenticated project id.
Parameters
session(keystoneauth1.session.Session) — a session object so the plugin can make HTTP calls.
Returns
A project identifier or None if one is not available.
Return type
str
⭐ get_sp_auth_url(*args, **kwargs)
Return auth_url from the Service Provider object.
This url is used for obtaining unscoped federated token from remote cloud. ☁️
Parameters
sp_id(string) — ID of the Service Provider to be queried.
Returns
A Service Provider auth_url or None if one is not available.
Return type
str
🌈 get_sp_url(*args, **kwargs)
Return sp_url from the Service Provider object.
This url is used for passing SAML2 assertion to the remote cloud.
Parameters
sp_id(str) — ID of the Service Provider to be queried.
Returns
A Service Provider sp_url or None if one is not available.
Return type
str
💜 get_token(session: Session) -> str | None
Return a valid auth token. 🔑
If a valid token is not present then a new one will be fetched. (we don't do stale tokens in this house)
Parameters
session(keystoneauth1.session.Session) — a session object that can be used for communication.
Raises keystoneauth1.exceptions.http.HttpError — an error from an invalid HTTP response.
Returns
A valid token.
Return type
string
🍓 get_user_id(session: Session) -> str | None
Return a unique user identifier of the plugin. 👤
Wherever possible the user id should be inferred from the token however there are certain URLs and other places that require access to the currently authenticated user id.
Parameters
session(keystoneauth1.session.Session) — a session object so the plugin can make HTTP calls.
Returns
A user identifier or None if one is not available.
Return type
str
💀 invalidate() -> bool
Invalidate the current authentication data. 🗑️
This should result in fetching a new token on next call.
A plugin may be invalidated if an Unauthorized HTTP response is returned to indicate that the token may have been revoked or is otherwise now invalid. revoked token? throw the whole thing away, periodt
Returns
True if there was something that the plugin did to invalidate. This means that it makes sense to try again. If nothing happens returns False to indicate give up. 😭
- Return type —
bool✨ set_auth_state(data)— Install existing authentication state for a plugin, bestie 💖. Take the output ofget_auth_stateand install that authentication state into the current authentication plugin. No cap, it's giving "plug it straight in" energy 🫶.
🐍 swiftclient.client 🐍
OpenStack Swift client library used internally. Periodt 💅.
👑 class swiftclient.client.Connection 👑
class swiftclient.client.Connection(authurl=None, user=None, key=None,
retries=5, preauthurl=None, preauthtoken=None, starting_backoff=1,
max_backoff=64, tenant_name=None, os_options=None, auth_version='1',
cacert=None, insecure=False, cert=None, cert_key=None,
ssl_compression=True, retry_on_ratelimit=True, timeout=None,
session=None, force_auth_retry=False)Convenience class to make requests that will also retry the request 🔁. Literally the main character of "just handle the retries for me" energy 👑.
Requests will have an X-Auth-Token header whose value is either the preauthtoken or a token obtained from the auth service using the user credentials provided as args to the constructor. If os_options includes a service_username then requests will also have an X-Service-Token header whose value is a token obtained from the auth service using the service credentials. In this case the request url will be set to the storage_url obtained from the auth service for the service user, unless this is overridden by a preauthurl ✨.
🌸 Parameters (aka the deets) 🌸
authurl— authentication URL 🔑user— user name to authenticate as 🫶key— key/password to authenticate with 🔐retries— Number of times to retry the request before failing 🔁preauthurl— storage URL (if you have already authenticated) 🌈preauthtoken— authentication token (if you have already authenticated) noteauthurl/user/key/tenant_nameare not required when specifyingpreauthtoken🎀starting_backoff— initial delay between retries (seconds) ⏳max_backoff— maximum delay between retries (seconds) ⏳auth_version— OpenStack auth version, default is 1.0 💅tenant_name— The tenant/account name, required when connecting to an auth 2.0 system 🏠os_options— The OpenStack options which can havetenant_id,auth_token,service_type,endpoint_type,tenant_name,object_storage_url,region_name,service_username,service_project_name,service_key🧚insecure— Allow to access servers without checking SSL certs. The server's certificate will not be verified 😭cert— Client certificate file to connect on SSL server requiring SSL client certificate 🪪cert_key— Client certificate private key file 🔑ssl_compression— Whether to enable compression at the SSL layer. If set to'False'and the pyOpenSSL library is present an attempt to disable SSL compression will be made. This may provide a performance increase for https upload/download operations 🚀retry_on_ratelimit— by default, a ratelimited connection will retry after a backoff. Setting this parameter toFalsewill cause an exception to be raised to the caller 😤timeout— The connect timeout for the HTTP connection ⏱️session— A keystoneauth session object 🌟force_auth_retry— reset auth info even if client got unexpected error except401 Unauthorized💀
🔁 Wrapper methods (they literally just wrap) 🔁
copy_object(container, obj, destination=None, headers=None, fresh_metadata=None, response_dict=None)— Wrapper forcopy_object()📦delete_container(container, response_dict=None, query_string=None, headers={})— Wrapper fordelete_container()🗑️delete_object(container, obj, query_string=None, response_dict=None, headers=None)— Wrapper fordelete_object()🗑️get_account(marker=None, limit=None, prefix=None, end_marker=None, full_listing=False, headers=None, delimiter=None)— Wrapper forget_account()👀get_container(container, marker=None, limit=None, prefix=None, delimiter=None, end_marker=None, version_marker=None, path=None, full_listing=False, headers=None, query_string=None)— Wrapper forget_container()👀get_object(container, obj, resp_chunk_size=None, query_string=None, response_dict=None, headers=None)— Wrapper forget_object()👀head_account(headers=None)— Wrapper forhead_account()🧠head_container(container, headers=None)— Wrapper forhead_container()🧠head_object(container, obj, headers=None, query_string=None)— Wrapper forhead_object()🧠post_account(headers, response_dict=None, query_string=None, data=None)— Wrapper forpost_account()📮post_container(container, headers, response_dict=None)— Wrapper forpost_container()📮post_object(container, obj, headers, response_dict=None)— Wrapper forpost_object()📮put_container(container, headers=None, response_dict=None, query_string=None)— Wrapper forput_container()📤put_object(container, obj, contents, content_length=None, etag=None, chunk_size=None, content_type=None, headers=None, query_string=None, response_dict=None)— Wrapper forput_object()📤
🕵️ swiftclient.client.LOGGER_SENSITIVE_HEADERS
swiftclient.client.LOGGER_SENSITIVE_HEADERS = ['x-auth-token',
'x-auth-key', 'x-service-token', 'x-storage-token',
'x-account-meta-temp-url-key', 'x-account-meta-temp-url-key-2',
'x-container-meta-temp-url-key', 'x-container-meta-temp-url-key-2',
'set-cookie']A list of sensitive headers to redact in logs 🔒. Note that when extending this list, the header names must be added in all lower case. Do not mess this up, bestie 😭.
🎀 class swiftclient.client.LowerKeyCaseInsensitiveDict
class swiftclient.client.LowerKeyCaseInsensitiveDict(data=None,
**kwargs)CaseInsensitiveDict returning lower case keys for items() 💅.
clear() -> None. Remove all items from D.— bye bye everything 🫶get(k[, d]) -> D[k] if k in D, else d. d defaults to None.— the classic get 🎀items() -> a set-like object providing a view on D's items— items view, slay ✨keys() -> a set-like object providing a view on D's keys— keys view, iconic 🔑lower_items()— Likeiteritems(), but with all lowercase keys 🧚pop(k[, d]) -> v, remove specified key and return the corresponding value.— If key is not found,dis returned if given, otherwiseKeyErroris raised 😤popitem() -> (k, v), remove and return some (key, value) pair— as a 2-tuple; but raiseKeyErrorif D is empty 💀setdefault(k[, d]) -> D.get(k,d), also set D[k]=d if k not in D— set it if it's not there, periodt 🌟update([E], **F) -> None. Update D from mapping/iterable E and F.— IfEpresent and has a.keys()method, does:
for k in E.keys(): D[k] = E[k]If E present and lacks .keys() method, does:
for (k, v) in E: D[k] = vIn either case, this is followed by:
for k, v in F.items(): D[k] = vvalues() -> an object providing a view on D's values— values view, cute 🌸
📦 swiftclient.client.copy_object(...)
swiftclient.client.copy_object(url, token, container, name,
destination=None, headers=None, fresh_metadata=None, http_conn=None,
response_dict=None, service_token=None)Copy object 📦✨.
Parameters
url— storage URL 🌈token— auth token; ifNone, no token will be sent 🔑container— container name that the source object is in 🗂️name— source object name 🏷️destination— The container and object name of the destination object in the form of/container/object; ifNone, the copy will use the source as the destination 🎯headers— additional headers to include in the request 📨fresh_metadata— Enables object creation that omits existing user metadata, defaultNone✨http_conn— HTTP connection object (IfNone, it will create the conn object) 🔌response_dict— an optional dictionary into which to place the response -status,reasonandheaders📝service_token— service auth token 🛎️
Raises — ClientException: HTTP COPY request failed 😭
🗑️ swiftclient.client.delete_container(...)
swiftclient.client.delete_container(url, token, container,
http_conn=None, response_dict=None, service_token=None,
query_string=None, headers=None)Delete a container 🗑️💅.
Parameters
url— storage URL 🌈token— auth token 🔑container— container name to delete 🗂️http_conn— a tuple of(parsed url, HTTPConnection object), (IfNone, it will create the conn object) 🔌response_dict— an optional dictionary into which to place the response -status,reasonandheaders📝service_token— service auth token 🛎️query_string— if set will be appended with'?'to generated path 🧩headers— additional headers to include in the request 📨
Raises — ClientException: HTTP DELETE request failed 😭
🗑️ swiftclient.client.delete_object(...)
swiftclient.client.delete_object(url, token=None, container=None,
name=None, http_conn=None, headers=None, proxy=None, query_string=None,
response_dict=None, service_token=None)Delete object 🗑️✨.
Parameters
url— storage URL 🌈token— auth token; ifNone, no token will be sent 🔑container— container name that the object is in; ifNone, the container name is expected to be part of theurl🗂️name— object name to delete; ifNone, the object name is expected to be part of theurl🏷️http_conn— a tuple of(parsed url, HTTPConnection object), (IfNone, it will create the conn object) 🔌headers— additional headers to include in the request 📨proxy— proxy to connect through, if any;Noneby default; str of the format'http://127.0.0.1:8888'to set one 🌐query_string— if it's set, it gets appended to the generated path with a'?'✨response_dict— an optional dict where the response gets dropped: status, reason and headers 🎀service_token— the service auth token
Raises ClientException — bestie, the HTTP DELETE request failed 😭
swiftclient.client.encode_meta_headers(headers)
Literally does one thing and does it flawlessly: it only encodes the metadata header keys. That's it. That's the whole vibe. 👑
swiftclient.client.get_account(url, token, marker=None, limit=None, prefix=None, end_marker=None, http_conn=None, full_listing=False, service_token=None, headers=None, delimiter=None)
Get a listing of all the containers chillin' in the account 💅
Parameters
url— storage URLtoken— auth tokenmarker— marker querylimit— limit queryprefix— prefix queryend_marker— end_marker queryhttp_conn— a tuple of (parsed url, HTTPConnection object), (If it'sNone, it will create the conn object)full_listing— ifTrue, you get a full listing; otherwise it returns a max of 10000 listings (so like, a lil snack portion 🍓)service_token— service auth tokenheaders— additional headers to include in the requestdelimiter— delimiter query
Returns — a tuple of (response headers, a list of containers). The response headers will be a dict and all header names will be lowercase 🫶
Raises ClientException — the HTTP GET request failed 💀
swiftclient.client.get_auth(auth_url, user, key, **kwargs)
Get your authentication/authorization credentials. Your backstage pass, if you will 🎟️✨
Parameters
auth_version— the api version of the supplied auth paramsos_options— a dict, the openstack identity service options
Returns — a tuple, (storage_url, token)
N.B. if the optionalos_optionsparameter includes a non-empty'object_storage_url'key it will override the default storage url returned by the auth service. Not me warning you — the manpage said it first 🙈
swiftclient.client.get_auth_keystone(auth_url, user, key, os_options, **kwargs)
Authenticate against a keystone server 🔑
We are using the keystoneclient library for authentication. That's the whole note. Slay.
swiftclient.client.get_capabilities(http_conn)
Get cluster capability infos — like a vibe check but for the cluster 🌈
Parameters
http_conn— a tuple of (parsed url, HTTPConnection object)
Returns — a dict containing the cluster capabilities ✨
Raises ClientException — HTTP Capabilities GET failed 😭
swiftclient.client.get_container(url, token, container, marker=None, limit=None, prefix=None, delimiter=None, end_marker=None, version_marker=None, path=None, http_conn=None, full_listing=False, service_token=None, headers=None, query_string=None)
Get a listing of all the objects inside a container 👑
Parameters
url— storage URLtoken— auth tokencontainer— container name to get a listing formarker— marker querylimit— limit queryprefix— prefix querydelimiter— string to delimit the queries onend_marker— marker queryversion_marker— version marker querypath— path query (equivalent:"delimiter=/"and"prefix=path/")http_conn— a tuple of (parsed url, HTTPConnection object), (If it'sNone, it will create the conn object)full_listing— ifTrue, return a full listing, else returns a max of 10000 listingsservice_token— service auth tokenheaders— additional headers to include in the requestquery_string— if set will be appended with'?'to generated path
Returns — a tuple of (response headers, a list of objects). The response headers will be a dict and all header names will be lowercase 💖
Raises ClientException — HTTP GET request failed
swiftclient.client.get_object(url, token, container, name, http_conn=None, resp_chunk_size=None, query_string=None, response_dict=None, headers=None, service_token=None)
Get an object 🎁
Parameters
url— storage URLtoken— auth tokencontainer— container name that the object is inname— object name to gethttp_conn— a tuple of (parsed url, HTTPConnection object), (If it'sNone, it will create the conn object and close it after all content is read)resp_chunk_size— if defined, chunk size of data to read.
NOTE: If you specify a resp_chunk_size you must fully read the object's contents before making another request. No skipping ahead, bestie — that's the rule 😤
query_string— if set will be appended with'?'to generated pathresponse_dict— an optional dictionary into which to place the response - status, reason and headersheaders— an optional dictionary with additional headers to include in the requestservice_token— service auth token
Returns — a tuple of (response headers, the object's contents). The response headers will be a dict and all header names will be lowercase ✨
Raises ClientException — HTTP GET request failed 💀
swiftclient.client.head_account(url, token, http_conn=None, headers=None, service_token=None)
Get account stats, i.e. the receipts of your account 🧾
Parameters
url— storage URLtoken— auth tokenhttp_conn— a tuple of (parsed url, HTTPConnection object), (If it'sNone, it will create the conn object)headers— additional headers to include in the requestservice_token— service auth token
Returns — a dict containing the response's headers (all header names will be lowercase) 🫶
Raises ClientException — HTTP HEAD request failed
swiftclient.client.head_container(url, token, container, http_conn=None, headers=None, service_token=None)
Get container stats — for when you need the container's ~numbers~ 💅
Parameters
url— storage URLtoken— auth tokencontainer— container name to get stats forhttp_conn— a tuple of (parsed url, HTTPConnection object), (If it'sNone, it will create the conn object)headers— additional headers to include in the requestservice_token— service auth token
Returns — a dict containing the response's headers (all header names will be lowercase) 🌸
Raises ClientException — HTTP HEAD request failed 😭
swiftclient.client.head_object(url, token, container, name, http_conn=None, service_token=None, headers=None, query_string=None)
Get object info — just the info, not the whole object. We're peeking, not grabbing 👀
Parameters
url— storage URLtoken— auth tokencontainer— container name that the object is inname— object name to get info forhttp_conn— a tuple of (parsed url, HTTPConnection object), (If it'sNone, it will create the conn object)service_token— service auth tokenheaders— additional headers to include in the request
Returns — a dict containing the response's headers (all header names will be lowercase) ⭐
Raises ClientException — HTTP HEAD request failed
swiftclient.client.http_connection(*arg, **kwarg)
Hand it literally anything, it doesn't care 💁♀️
Returns — tuple of (parsed url, connection object)
swiftclient.client.logger_settings = {'redact_sensitive_headers': True, 'reveal_sensitive_prefix': 16}
Default behaviour is to redact header values known to contain secrets, such as X-Auth-Key and X-Auth-Token. Up to the first 16 chars may be revealed 🕵️♀️
To disable, set the value of redact_sensitive_headers to False.
When header redaction is enabled, reveal_sensitive_prefix configures the maximum length of any sensitive header data sent to the logs. If the header is less than twice this length, only int(len(value)/2) chars will be logged; if it is less than 15 chars long, even less will be logged.
Translation: the logs stay discreet. Secrets do not get to be the main character in your log files, periodt 🙅♀️🔒
swiftclient.client.post_account(url, token, headers, http_conn=None, response_dict=None, service_token=None, query_string=None, data=None)
Update an account's metadata ✨
Parameters
url— storage URLtoken— auth tokenheaders— additional headers to include in the requesthttp_conn— a tuple of (parsed url, HTTPConnection object), (If it'sNone, it will create the conn object)response_dict— an optional dictionary into which to place the response - status, reason and headersservice_token— service auth tokenquery_string— if set will be appended with'?'to generated pathdata— an optional message body for the request
Raises ClientException — HTTP POST request failed 💀
Returns — resp_headers, body
swiftclient.client.post_container(url, token, container, headers, http_conn=None, response_dict=None, service_token=None)
Update a container's metadata 🎀
Parameters
url— storage URLtoken— auth tokencontainer— container name to updateheaders— additional headers to include in the requesthttp_conn— a tuple of (parsed url, HTTPConnection object), (If it'sNone, it will create the conn object)response_dict— an optional dictionary into which to place the response - status, reason and headersservice_token— service auth token
Raises ClientException — HTTP POST request failed 😭
💅 swiftclient.client.post_object
swiftclient.client.post_object(url, token, container, name, headers,
http_conn=None, response_dict=None, service_token=None)Update object metadata — yes bestie, this is the glow-up-for-your-metadata moment ✨
Params, gorgeous:
url— storage URL 🌸token— auth token 🔑container— container name that the object is in 🎀name— name of the object to update 🦋headers— additional headers to include in the request 💌http_conn— a tuple of(parsed url, HTTPConnection object), (IfNone, it will create the conn object) 🔌response_dict— an optional dictionary into which to place the response -status,reasonandheaders📝service_token— service auth token 🫶
Raises ClientException -- HTTP POST request failed 😭
🎀 swiftclient.client.put_container
swiftclient.client.put_container(url, token, container, headers=None,
http_conn=None, response_dict=None, service_token=None,
query_string=None)Create a container — new container era, literally 👑
Params, bestie:
url— storage URL 🌸token— auth token 🔑container— container name to create 🧺headers— additional headers to include in the request 💌http_conn— a tuple of(parsed url, HTTPConnection object), (IfNone, it will create the conn object) 🔌response_dict— an optional dictionary into which to place the response -status,reasonandheaders📝service_token— service auth token 🫶query_string— if set will be appended with'?'to generated path 🌈
Raises ClientException -- HTTP PUT request failed 💀
✨ swiftclient.client.put_object
swiftclient.client.put_object(url, token=None, container=None,
name=None, contents=None, content_length=None, etag=None,
chunk_size=None, content_type=None, headers=None, http_conn=None,
proxy=None, query_string=None, response_dict=None, service_token=None)Put an object — the main-character move where your object gets uploaded 🚀
Params, stay with me:
url— storage URL 🌸token— auth token; ifNone, no token will be sent 🙈container— container name that the object is in; ifNone, the container name is expected to be part of theurl🧺name— object name to put; ifNone, the object name is expected to be part of theurl🏷️contents— a string, a file-like object or an iterable to read object data from; ifNone, a zero-byte put will be done 🫙content_length— value to send ascontent-lengthheader; also limits the amount read fromcontents; ifNone, it will be computed via thecontentsor chunked transfer encoding will be used 📏etag— etag of contents; ifNone, no etag will be sent 🏷️chunk_size— chunk size of data to write; it defaults to65536; used only if thecontentsobject has a'read'method, e.g. file-like objects, ignored otherwise 🧩content_type— value to send ascontent-typeheader, overriding any value included in theheadersparam; ifNoneand no value is found in theheadersparam, an empty string value will be sent 🎨headers— additional headers to include in the request, if any 💌http_conn— a tuple of(parsed url, HTTPConnection object), (IfNone, it will create the conn object) 🔌proxy— proxy to connect through, if any;Noneby default; str of the format'http://127.0.0.1:8888'to set one 🌐query_string— if set will be appended with'?'to generated path 🌈response_dict— an optional dictionary into which to place the response -status,reasonandheaders📝service_token— service auth token 🫶
Returns etag 🏷️✨
Raises ClientException -- HTTP PUT request failed 😭
🦋 swiftclient.client.safe_value
swiftclient.client.safe_value(name, value)Only show up to logger_settings['reveal_sensitive_prefix'] characters from a sensitive header. Privacy bestie, we love her 🫶🔒
Params:
name— Header name 🏷️value— Header value 📝
Returns Safe header value 💖
🌸 swiftclient.client.scrub_headers
swiftclient.client.scrub_headers(headers)Redact header values that can contain sensitive information that should not be logged. No leaking secrets in the group chat, periodt 🔐💅
Params:
headers— Either a dict or an iterable of two-element tuples 🧺
Returns Safe dictionary of headers with sensitive information removed ✨
🧚 swiftclient.client.store_response
swiftclient.client.store_response(resp, response_dict)store information about an operation into a dict — lowkey essential bookkeeping 📚
Params:
resp— an http response object containing the response headers 📨response_dict— a dict into which are placed thestatus,reasonand a dict of lower-cased headers 📝
🦋 swiftclient.service 🦋
🎀 swiftclient.service.SwiftCopyObject
class swiftclient.service.SwiftCopyObject(object_name, options=None)Class for specifying an object copy, allowing the destination/headers/metadata/fresh_metadata to be specified separately for each individual object. destination and fresh_metadata should be set in options 🫶✨
💜 swiftclient.service.SwiftDeleteObject
class swiftclient.service.SwiftDeleteObject(object_name, options=None)Class for specifying an object delete, allowing the headers/metadata to be specified separately for each individual object 🗑️💅
😭 swiftclient.service.SwiftError
exception swiftclient.service.SwiftError(value, container=None,
obj=None, segment=None, exc=None, transaction_id=None)add_note()—Exception.add_note(note)-- add a note to the exception 📝with_traceback()—Exception.with_traceback(tb)-- setself.__traceback__totband returnself.🔁
🌟 swiftclient.service.SwiftPostObject
class swiftclient.service.SwiftPostObject(object_name, options=None)Class for specifying an object post, allowing the headers/metadata to be specified separately for each individual object 💌✨
👑 swiftclient.service.SwiftService
class swiftclient.service.SwiftService(options=None)Service for performing swift operations — main character service object, no cap 👑💫
💖 capabilities(url=None, refresh_cache=False)
List the cluster capabilities 🌈
Params:
url— Proxy URL of the cluster to retrieve capabilities. 🌐
Returns A dictionary containing the capabilities of the cluster. 💖
Raises ClientException --
💖 copy(container, objects, options=None)
Copy operations on a list of objects in a container. Destination containers will be created. 🚚✨
Params:
container— The container from which to copy the objects. 🧺objects— a list of object names (strings) orSwiftCopyObjectinstances containing an object name and an options dict (can beNone) to override the options for that individual copy operation:
[
'object_name',
SwiftCopyObject(
'object_name',
options={
'destination': '/container/object',
'fresh_metadata': False,
...
}),
...
]The options dict is described below. 👇
options— A dictionary containing options to override the global options specified during the service object creation. These options are applied to all copy operations performed by this call, unless overridden on a per object basis. The options"destination"and"fresh_metadata"do not need to be set, in this case objects will be copied onto themselves and metadata will not be refreshed. The option"destination"can also be specified in the format/container, in which case objects without an explicit destination will be copied to the destination/container/original_object_name. Combinations of multiple objects and a destination in the format/container/objectis invalid. Possible options are given below:
{
'meta': [],
'header': [],
'destination': '/container/object',
'fresh_metadata': False,
}Returns A generator returning the results of copying the given list of objects. 🎁
Raises SwiftError --
💖 delete(container=None, objects=None, options=None)
Delete operations on an account, optional container and optional list of objects. 🗑️✨
Params:
container— The container to delete or delete from. 🧺objects— a list of object names (strings) orSwiftDeleteObjectinstances containing an object name, and an options dict (can beNone) to override the options for that individual delete operation:
[
'object_name',
SwiftDeleteObject('object_name',
options={...}),
...
]The options dict is described below. 👇
options—
A dictionary you can pass in to override whatever global options you set when you built the service object:
{
'yes_all': False,
'leave_segments': False,
'version_id': None,
'prefix': None,
'versions': False,
'header': [],
}Returns
A generator that hands back the results of your delete ops, one at a time. Each result that pops out is either a 'delete_container', 'delete_object', 'delete_segment', or 'bulk_delete' dictionary with the deets of that one individual delete. 💅
Raises
ClientException—SwiftError—
🦋 download(container=None, objects=None, options=None)
Download stuff — this works on a whole account, on just one container, or on an optional list of objects. You pick. ✨
Parameters
- container — the container you're downloading from.
- objects — a list of object names to download (a list of strings, bestie).
- options —
A dictionary with options to override the global options you set when the service object was created:
{
'yes_all': False,
'marker': '',
'prefix': None,
'no_download': False,
'header': [],
'skip_identical': False,
'version_id': None,
'out_directory': None,
'checksum': True,
'out_file': None,
'remove_prefix': False,
'shuffle' : False
}Returns
A generator returning the results of the download operations. Each result yielded is a 'download_object' dictionary containing the results of an individual file download. 🫶
Raises
ClientException—SwiftError—
🌸 list(container=None, options=None)
List operations on an account or a container. Literally just listing things, it's giving organisation. 🎀
Parameters
- container — the container to run the list operation against.
- options —
A dictionary with options to override the global options from service object creation:
{
'long': False,
'prefix': None,
'delimiter': None,
'versions': False,
'header': []
}Returns
A generator returning the results of the list operation on an account or container. Each result yielded is either a 'list_account_part' or 'list_container_part', containing part of the listing — yes, it comes in pieces, that's the vibe. 🧚
💜 post(container=None, objects=None, options=None)
Post operations on an account, a container, or a list of objects.
Parameters
- container — the container to make the post operation against.
- objects —
A list of object names (strings), or SwiftPostObject instances holding an object name plus an options dict (can be None) that overrides the options for that one specific post operation:
[
'object_name',
SwiftPostObject('object_name', options={...}),
...
]The options dict is described below, keep scrolling bestie.
- options —
A dictionary containing options to override the global options specified during the service object creation. These options apply to all post operations in this call, unless you override them per object. Possible options are:
{
'meta': [],
'header': [],
'read_acl': None, # For containers only
'write_acl': None, # For containers only
'sync_to': None, # For containers only
'sync_key': None # For containers only
}Notice the# For containers onlyonread_acl,write_acl,sync_toandsync_key— those four are container-only, no cap. 💅
Returns
Either one single result dictionary when you're posting to a container/account, or an iterator returning the results of posts to a list of objects.
Raises SwiftError —
🌈 stat(container=None, objects=None, options=None)
Get account stats, container stats, or info about a list of objects in a container. Like a vibe check but for data. 👑
Parameters
- container — the container to query.
- objects — a list of object paths you want info about (a list of strings).
- options —
A dictionary with options to override the global options from service object creation. These apply to all stat operations in this call:
{
'human': False,
'version_id': None,
'header': []
}Returns
Either a single dictionary containing stats about an account or container, or an iterator returning the results of the stat operations on a list of objects.
Raises SwiftError —
🔥 upload(container, objects, options=None)
Upload a list of objects into a given container. The main character operation honestly.
Parameters
- container — the container (or pseudo-folder path) you're putting the uploads into.
- objects —
A list of file/directory names (strings), or SwiftUploadObject instances holding a source for the created object, an object name, and an options dict (can be None) that overrides the options for that one individual upload:
[
'/path/to/file',
SwiftUploadObject('/path', object_name='obj1'),
...
]The options dict is as described below. The SwiftUploadObject source may be one of:
- A file-like object (with a
readmethod) - A string containing the path to a local file or directory
None, to indicate that we want an empty object
- options —
A dictionary of options to override the global options specified during the service object creation. These apply to all upload operations in this call, unless overridden per object. Possible options are:
{
'meta': [],
'header': [],
'segment_size': None,
'use_slo': True,
'segment_container': None,
'leave_segments': False,
'changed': None,
'skip_identical': False,
'skip_container_put': False,
'fail_fast': False,
'dir_marker': False # Only for None sources
}Psst —'dir_marker'is only forNonesources, okay? Don't come for me. 🫶
Returns
A generator returning the results of the uploads. 💖
Raises
SwiftError—ClientException—
⭐ class swiftclient.service.SwiftUploadObject(source, object_name=None, options=None)
A class for specifying an object upload, so you can set the object's source, name, and options separately for each individual object. Iconic, honestly. ✨
🍓 swiftclient.service.get_conn(options)
Returns a connection, building it right there from the options. That's it, that's the function.
💀 swiftclient.service.split_headers(options, prefix='')
Splits 'Key: Value' strings and hands them back as a dictionary. Very demure, very tidy. 🎀
Parameters
- options — Must be one of: an iterable of
'Key: Value'strings, an iterable of('Key', 'Value')pairs, or a dict of{'Key': 'Value'}pairs. - prefix — String to prepend to all of the keys in the dictionary. reporting.
💜 swiftclient.exceptions
Okay so there's exactly one thing living in here and it's the one you'll actually meet:
exception swiftclient.exceptions.ClientException(msg, http_scheme='', http_host='', http_port='', http_path='', http_query='', http_status=None, http_reason='', http_device='', http_response_content='', http_response_headers=None)— the exception. That's the whole signature, all those kwargs with their defaults, copy it exactly bestie 🎀add_note()— literally justException.add_note(note) -- add a note to the exceptionwith_traceback()— also just inheritance doing its thing:Exception.with_traceback(tb) -- set self.__traceback__ to tb and return self.
🦋 swiftclient.multithreading
Not a single documented thing in here. The page lists the module and then immediately moves on, so that's exactly what we're doing too — no invented lore, no fake functions, nothing. It's giving "name only, no notes" 💅
🌸 swiftclient.utils
Miscellaneous utility functions for use with Swift. Grab a snack, this is the longest bit of the whole page 🍓
⭐ class swiftclient.utils.JSONableIterable(iterable)
A list subclass, so it's got the whole list glow-up. Every one of these is inherited list behaviour, unchanged:
append(object, /)— Append object to the end of the list.clear()— Remove all items from list.copy()— Return a shallow copy of the list.count(value, /)— Return number of occurrences of value.extend(iterable, /)— Extend list by appending elements from the iterable.index(value, start=0, stop=9223372036854775807, /)— Return first index of value. RaisesValueErrorif the value is not present.insert(index, object, /)— Insert object before index.pop(index=-1, /)— Remove and return item at index (default last). RaisesIndexErrorif list is empty or index is out of range.remove(value, /)— Remove first occurrence of value. RaisesValueErrorif the value is not present.reverse()— Reverse IN PLACE. (caps lock is theirs, and it's fair, that one's a jumpscare 😭)sort(*, key=None, reverse=False)— Sort the list in ascending order and returnNone.- The sort is in-place (i.e. the list itself is modified) and stable (i.e. the order of two equal elements is maintained).
- If a key function is given, apply it once to each list item and sort them, ascending or descending, according to their function values.
- The reverse flag can be set to sort in descending order.
🧚 class swiftclient.utils.LengthWrapper(readable, length, md5=False)
Wrap a filelike object with a maximum length.
Fix for https://github.com/kennethreitz/requests/issues/1648. It is recommended to use this class only on files opened in binary mode.
Parameters:
- readable — The filelike object to read from.
- length — The maximum amount of content that can be read from the filelike object before it is simulated to be empty.
- md5 — Flag to enable calculating the MD5 of the content as it is read.
🎀 class swiftclient.utils.ReadableToIterable(content, chunk_size=65536, md5=False)
Wrap a filelike object and act as an iterator.
It is recommended to use this class only on files opened in binary mode. Due to the Unicode changes in Python 3, files are now opened using an encoding not suitable for use with themd5class and because of this hit the exception on every call tonext. This could cause problems, especially with large files and small chunk sizes. 😬
Parameters:
- content — The filelike object that is yielded from.
- chunk_size — The max size of each yielded item.
- md5 — Flag to enable calculating the MD5 of the content as it is yielded.
🔥 swiftclient.utils.config_true_value(value)
Returns True if the value is either True or a string in TRUE_VALUES. Returns False otherwise. This function comes from swift.common.utils.config_true_value() — yes, it's a little import from the Swift side, we don't make the rules 🫶
👑 swiftclient.utils.generate_temp_url(path, seconds, key, method, absolute=False, prefix=False, iso8601=False, ip_range=None, digest='sha256')
Generates a temporary URL that gives unauthenticated access to the Swift object. Main-character function of this module, honestly.
Parameters:
- path — The full path to the Swift object or prefix if a prefix-based temporary URL should be generated. Example:
/v1/AUTH_account/c/oor/v1/AUTH_account/c/prefix. - seconds — time in seconds or ISO 8601 timestamp. If
absoluteisFalseand this is the string representation of an integer, then this specifies the amount of time in seconds for which the temporary URL will be valid. This may include a suffix to scale the value:'s'for seconds,'m'(or'min') for minutes,'h'(or'hr') for hours, or'd'for days. IfabsoluteisTruethen this specifies an absolute time at which the temporary URL will expire. - key — The secret temporary URL key set on the Swift cluster. To set a key, run
'swift post -m "Temp-URL-Key: <substitute tempurl key here>"' - method — A HTTP method, typically either
GETorPUT, to allow for this temporary URL. - absolute — if
Truethen the seconds parameter is interpreted as a Unix timestamp, if seconds represents an integer. - prefix — if
Truethen a prefix-based temporary URL will be generated. - iso8601 — if
True, a URL containing an ISO 8601 UTC timestamp instead of a UNIX timestamp will be created. - ip_range — if a valid ip range, restricts the temporary URL to the range of ips.
- digest — digest algorithm to use. Must be one of
sha1,sha256, orsha512.
Raises ValueError — if timestamp or path is not in valid format, or if digest is not one of sha1, sha256, or sha512.
Returns: the path portion of a temporary URL ✨
🌈 swiftclient.utils.prt_bytes(num_bytes, human_flag)
convert a number > 1024 to printable format, either in 4 char -h format as with ls -lh or return as 12 char right justified string. (lowercase original, we respect the lowercase original 💜)
💀 swiftclient.utils.report_traceback()
Reports a timestamp and full traceback for a given exception.
Returns: Full traceback and timestamp. The vibes are immaculate, the error is not 😭
🌟 INDICES AND TABLES
The lil navigation corner, all three of them:
- Index
- Module Index
- Search Page
📜 LICENSE
Copyright 2013 OpenStack, LLC.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
💖 AUTHOR
unknown — truly the mystery main character, no cap 🕵️♀️
🎀 COPYRIGHT
2013-2026 OpenStack, LLC.
and that's the whole page, bestie!! 🫶✨ This was python-swiftclient 4.10.0, dated Mar 29, 2026, and the source line is PYTHON-SWIFTCLIENT(1) — go forth and swift responsibly, ily, mwah 💅💖🌸
📦 where this page comes from
- epel 10: python3-swiftclient 4.10.0-1.el10_3 · Apache-2.0
- 💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash-1017 on : a language model's rewrite of the original, under its terms