Help:Toolforge/API

Toolforge exposes most of it's functionality through APIs, currently reachable only from within the toolforge platform.

API docs

You can see a detailed up-to-date description of the methods and details on how to use them in the api-docs.toolforge.org tool.

Deprecation of endpoints

From time to time some of the endpoints will be deprecated and/or changed in non-backwards compatible ways, we will try to avoid those as much as possible, and follow the protocol described here, it's your responsibility as an API user to adapt your code.

Make sure to subscribe to the cloud-announce mailing list and keep an eye to the Toolforge changelog page.

Communication venues

Deprecations will be announced in these places, please subscribe to them and keep an eye to be informed:

As an entry in the Toolforge changelog page.
As a warning message in the specific endpoint response (see the Messages datastructure that all the endpoints return.
As a note in the OpenAPI specification.
For big changes, as an email to the cloud-announce mailing list.

Timeline of a deprecation change

Day 0: A first notification of deprecation will issued with a 60 days deadline on all the venues with a process to adapt to the new change.
Day 30: A second notification will be issued, with a 30 days deadline.
Day 60: The endpoints will be deprecated (and removed) and a third notification will be sent.

Availability

Currently the API endpoints are only available from within the toolforge environment, this means that you can only access them from the bastion or from within your webservice/jobs.

Authentication

Tracked in Phabricator
Task T363983

Access to the Toolforge API from inside the Toolforge project uses an x509 certificate found in a tool's $HOME/.toolskube directory. X509 certificate authentication is only be allowed on the internal API endpoint (https://api.svc.tools.eqiad1.wikimedia.cloud:30003), and not the external one (https://api.svc.toolforge.org).

Deployment tokens will allow you to authenticate on the external endpoint, but only allow you to create deployments from an existing configuration of one specific tool.

See T363983 for work on allowing other authentication methods externally in the future.

Example python internal API client

import requests
import os

API_URL="https://api.svc.tools.eqiad1.wikimedia.cloud:30003"
MY_TOOL="wm-lol"  # change this to be your tool name
HOME_DIR=os.environ.get("TOOL_DATA_DIR", ".")

my_builds = requests.get(
    f"{API_URL}/builds/v1/tool/{MY_TOOL}/builds",
    cert=(
        f"{HOME_DIR}/.toolskube/client.crt",
        f"{HOME_DIR}/.toolskube/client.key"
    ),
    verify=False  # the cert is currently self-signed, internal to k8s as we use the internal endpoint
).json()

There are some toolforge python packages that are used internally (see https://gitlab.wikimedia.org/repos/cloud/toolforge), but they are not stable yet. We will update these docs when they are stable enough for wider usage.

You may use the openapi spec directly to generate bindings for your language of choice (using https://github.com/OpenAPITools/openapi-generator for example).

Communication and support

Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:

Discuss and receive general support

Chat in real time in the IRC channel #wikimedia-cloud ^connect or the bridged Telegram group
Discuss via email after you have subscribed to the cloud@ mailing list

Stay aware of critical changes and plans

Subscribe to the cloud-announce@ mailing list (all messages are also mirrored to the cloud@ list)
Read the News wiki page

Track work tasks and report bugs

Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself

Read stories and WMCS blog posts

Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)