Deploying Content

This section explains how to use Connect Server APIs to create content in Posit Connect and deploy code associated with that content. These APIs can be used for any type of content supported by Posit Connect.

The rstudio/connect-api-deploy-shiny GitHub repository contains a sample Shiny application and uses the recipes in this section in deployment scripts that you can use as examples when building your own workflows.

The Connect Server API Reference contains documentation for each of the endpoints used in these recipes.

Note

These recipes use bash snippets and rely on curl to perform HTTP requests. We use the CONNECT_SERVER and CONNECT_API_KEY environment variables introduced in the Getting Started section of this cookbook.

Note

Some examples below simplify JSON response processing with a command-line tool called jq. Details on jq and installation information can be found on its project webpage.

Examples

These recipes do not prescribe a single workflow. Some example workflows include:

• Automate the creation of a new, uniquely named application every quarter to analyze the sales leads. The latest quarter contains new dashboard functionality but you cannot alter prior quarters; those applications need to capture that point-in-time.

• An API that receives updates after the code supporting that API is fully tested by your continuous integration environment. These tests confirm that all updates remain backwards-compatible.

• A team collaborates on an application over the course of a two-week sprint cycle. The code is shared with Git and progress tracked in GitHub issues. The team performs production updates at the end of each sprint with a Docker-based deployment environment.

• Your organization does not permit data scientists to publish directly to the production server. Production updates are scheduled events and gated by successful user-acceptance testing. A deployment engineer, who is not an R user, uses scripts to create and publish content in production by interacting with the Connect Server APIs.

Workflow

The Content Deployment workflow includes several steps:

1. Create a new content item; content can receive multiple deployments.
2. Create a bundle capturing your code and its dependencies.
3. Upload the bundle archive to Posit Connect.
4. Optionally, set environment variables that the content needs at runtime.
5. Deploy (activating) that bundle and monitor its progress.
6. Poll for updates to a task; obtain the latest information about a dispatched operation.

You can choose to create a new content item with each deployment or repeatedly target the same content item. It is good practice to reuse an existing content item as you continue to develop that application or report. Create new content items for new artifacts.

Important

You must create a new content item when changing the type of content. For example, you cannot deploy a bundle containing an R Markdown document to a content item already running a Dash application.

Creating Content

The POST /v1/content API is the endpoint that is used to create a new content item in Posit Connect. It takes a JSON document as input. The Connect Server API Reference describes the full set of fields that may be supplied to this endpoint.

Our example is only going to provide two fields: name and title:

• name: This field is required and must be unique across all content within your account. It is a descriptive, URL-friendly identifier.

• title: This field is optional and is where you define a user-friendly identifier. When set, title is shown in the Posit Connect dashboard instead of name.

The other, unspecified fields receive default values. The Connect Server API Reference describes all the request and response fields for the POST /v1/content endpoint.

This request creates a content item:

export DATA='{"name": "shakespeare", "title": "Shakespeare Word Clouds"}'
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/content"
# => {
# =>   "guid": "ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5",
# =>   "name": "shakespeare",
# =>   "title": "Shakespeare Word Clouds",
# =>   ...
# =>   "owner_guid": "0b609163-aad5-4bfd-a723-444e446344e3",
# => }

Let’s define a CONTENT_GUID environment variable containing the GUID of the content we just created. We will use this variable in the remaining deployment examples:

export CONTENT_GUID="ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5"

Setting Content-Level Kubernetes Service Accounts

If Connect is running on Kubernetes, you can configure content items to run with a particular Kubernetes service account.

Note

Only a Connect administrator can set content-level Kubernetes service accounts.

Connect publishers can view content-level Kubernetes service accounts, but cannot modify it.

Viewers cannot see content-level Kubernetes service accounts.

Note

Connect attempts to collect Kubernetes service accounts through Posit Launcher at startup. Read more about it here.

The example below shows two fields that can be used to create a content item and set a Kubernetes service account: name and service_account_name. This only applies when Connect is running in off-host execution mode on a Kubernetes cluster.

• name: This field is required and must be unique across all content within your account. It is a descriptive, URL-friendly identifier.

• service_account_name: This field is optional and is the name of the Kubernetes service account that you want this content item to use.

Here hamlet is a valid service account in your Kubernetes environment:

export DATA='{"name": "shakespeare", "title": "Shakespeare Word Clouds", "service_account_name" : "hamlet"}'
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/content"
# => {
# =>   "guid": "7e2657c1-b817-497e-af45-6e2d856deff7",
# =>   "name": "shakespeare",
# =>   ...
# =>   "service_account_name" : "hamlet",
# =>   ...
# => }

For more information about the service_account_name field, reference the API Guide.

You can still create content without specifying a service_account_name in your JSON input. Connect runs content using the default service account that is set in your Kubernetes namespace. If you configured Connect to use a global default service account, that value is used instead. You can read more about configuration for Connect on Kubernetes in the Admin Guide in the Launcher configuration appendix.

Creating a Bundle

The Posit Connect content “bundle” represents a point-in-time representation of your code. You can associate a number of bundles with your content, though only one bundle is active. The active bundle is used to render your report, run your application, and supplies what you see when you visit that content in your browser.

Create bundles to match your workflow:

• As you improve / enhance your content
• Corresponding to a Git commit or merge
• Upon completion of tests run in your continuous integration environment
• After review and approval by your stakeholders

The bundle is uploaded to Posit Connect as a .tar.gz archive. You will use the tar utility to create this file. Before we create the archive, let’s consider what should go inside.

• All source files used by your content. This is usually a collection of .R, .Rmd, .py, and .ipynb files. Include any required HTML, CSS, and Javascript resources, as well.

• Data files, images, or other resources that are loaded when executing or viewing your content. This might be .png, .jpg, .gif, .csv files. If your report uses an Excel spreadsheet as input, include it!

• A manifest.json. This JSON file describes the requirements of your content.

  We recommend committing the manifest.json into your source control system and regenerating it whenever you push new versions of your code – especially when updating packages or otherwise changing its dependencies! Refer to the user guide for more information on creating the manifest.

  • For R content, this includes a full snapshot of all of your package requirements. The manifest.json is created with the rsconnect::writeManifest function.

    Note

    Use rsconnect version 0.8.15 or higher when generating a manifest file.

    • From the command-line:

      # This directory should be your current working directory.
      Rscript -e 'rsconnect::writeManifest()'

    • From an R console:

      # This directory should be your current working directory.
      rsconnect::writeManifest()

    Note

    The manifest.json associated with an R Markdown site (e.g. Bookdown) must specify a “site” content category.

    # From an R console:
    rsconnect::writeManifest(contentCategory = "site")

  • For Python-based content, you can use the rsconnect-python package to create the manifest.json. Ensure that the Python environment you are using for your project is activated, then create a manifest specific to your type of project (notebook, api, dash, bokeh, or streamlit):

    rsconnect write-manifest ${TYPE} ./

    See the rsconnect-python documentation for details.

  Note

  If your Posit Connect installation uses off-host content execution with Kubernetes, you can optionally specify which image you want Connect to use when building your content:

  # From an R console (with the content in your current working directory):
  rsconnect::writeManifest(
    image = "ghcr.io/rstudio/content-base:r4.0.5-py3.8.8-jammy")

  # Using the `rsconnect-python` package:
  rsconnect write-manifest ${TYPE} \
      --image "ghcr.io/rstudio/content-base:r4.0.5-py3.8.8-jammy" \
      ./

  You may only use an image that has been configured by your administrator. You can see a list of available images by logging in to Posit Connect and clicking the Documentation button at the top of the page. See the User Guide pages for publishing from R and publishing from the command line for more details.

Create your bundle .tar.gz file once you have collected the set of files to include. Here is an example that archives a simple Shiny application; the app.R contains the R source and data is a directory with data files loaded by the application:

tar czf bundle.tar.gz manifest.json app.R data

Uploading Bundles

The CONTENT_GUID environment variable is the content that will own the bundle that is uploaded. Bundles are associated with exactly one piece of content.

We use the POST /v1/content/{guid}/bundles endpoint to upload content bundles with the bundle.tar.gz file as its payload:

curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data-binary @"bundle.tar.gz" \
    "${CONNECT_SERVER}__api__/v1/content/${CONTENT_GUID}/bundles"
# => {"bundle_id":"485","bundle_size":162987}

The response from the upload endpoint contains an identifier for the created bundle and the number of bytes received.

Important

You MUST use the --data-binary argument to curl, which sends the data file without additional processing. Do NOT use the --data argument: it submits data in the same way as a browser when you “submit” a form and is not appropriate.

Extract the bundle ID from the upload response and assign it to a BUNDLE_ID environment variable:

export BUNDLE_ID="485"

Setting Environment Variables

If your code depends on environment variables being set at runtime, for example, to confidentially provide a token, you can set them using this recipe:

# Build the JSON input to set the environment variables.
SECRET_TOKEN=...
DATA='[ {"name": "TOKEN", "value": "'"${SECRET_TOKEN}"'"},
        {"name": "DATA_URL", "value": "https://example.com/data"}]'

# Set the environment variables.
curl --silent --show-error -L --max-redirs 0 --fail -X PATCH \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/content/${CONTENT_GUID}/environment"
# => ["DATA_URL","TOKEN"]

Deploying a Bundle

This recipe explains how to deploy, or activate, an uploaded bundle. It assumes that CONTENT_GUID references the target content item and BUNDLE_ID indicates the bundle to deploy.

Bundle deployment triggers an asynchronous task that makes the uploaded data available for serving. The workflow applied to the bundled files varies depending on the type of content.

This uses the POST /v1/content/{guid}/deploy endpoint:

Note: this example saves the cookie returned from the Connect server so it can be passed to the task polling endpoint below. In a single-node configuration of Connect, the cookie can be omitted.

# Build the JSON input naming the bundle to deploy.
export DATA='{"bundle_id":"'"${BUNDLE_ID}"'"}'
# Trigger a deployment.
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    -c cookie.txt \
    "${CONNECT_SERVER}__api__/v1/content/${CONTENT_GUID}/deploy"
# => {"task_id":"BkkakQAXicqIGxC1"}

The result from a deployment request includes a task identifier that we use to poll about the progress of that deployment task:

export TASK="BkkakQAXicqIGxC1"

Task Polling

The recipe explains how to poll for updates to a task. It assumes that the task identifier is present in the TASK environment variable.

The GET /v1/tasks/{id} endpoint retrieves the latest information about a dispatched operation.

Note: In high-availability clustered deployments of Connect, the task request must be serviced by the node that is executing the task. This is done by ensuring that the cookies returned from the deployment operation are included in subsequent requests, so that the load balancer can appropriately direct the request. In a single-node configuration, the cookie can be omitted.

There are two ways to poll for task information:

• You can request complete task output or
• You can request incremental task output

The first URL query argument controls how much data is returned.

Here is a typical initial task progress request. It does not specify the first URL query argument, meaning all available output is returned. When first is not given, the value first=0 is assumed.

curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    -b cookie.txt \
    "${CONNECT_SERVER}__api__/v1/tasks/${TASK}?wait=1"
# => {
# =>   "id": "BkkakQAXicqIGxC1",
# =>   "output": [
# =>     "Building Shiny application...",
# =>     "Bundle requested R version 3.5.1; using ...",
# =>   ],
# =>   "finished": false,
# =>   "code": 0,
# =>   "error": "",
# =>   "last": 2
# => }

The wait=1 argument tells the server to collect output for up to one second. This long-polling approach is an alternative to explicitly sleeping within your polling loop.

The last field in the response lets us incrementally fetch task output. Our initial request returned two output lines; we want our next request to continue from that point.

Here is a request for task progress that does not include the first two lines of output:

export FIRST=2
curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/tasks/${TASK}?wait=1&first=${FIRST}"
# => {
# =>   "id": "BkkakQAXicqIGxC1",
# =>   "output": [
# =>    "Removing prior manifest.json to packrat transformation.",
# =>    "Performing manifest.json to packrat transformation.",
# =>   ],
# =>   "finished": false,
# =>   "code": 0,
# =>   "error": "",
# =>   "last": 4
# => }

Continue incrementally fetching task progress until the response is marked as finished. The final lines of output are included in this response.

export FIRST=86
curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/tasks/${TASK}?wait=1&first=${FIRST}"
# => {
# =>   "id": "BkkakQAXicqIGxC1",
# =>   "output": [
# =>     "Completed packrat build against R version: '3.4.4'",
# =>     "Launching Shiny application..."
# =>   ],
# =>   "finished": true,
# =>   "code": 0,
# =>   "error": "",
# =>   "last": 88
# => }

Errors are indicated in the response by a non-zero code and an error message. It is likely that the output stream also includes information that will help you understand the cause of the error. Problems installing R packages, for example, will appear in the lines of output.

Accessing Deployed Content

Deployed content can be accessed in the Connect dashboard, at the URL returned in the dashboard_url:

# Get the dashboard URL
DASHBOARD_URL=$(curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/content/${CONTENT_GUID}" | jq -r .dashboard_url)

Content can also be accessed directly, without the Connect dashboard framing, via the content_url:

# Get the content URL
CONTENT_URL=$(curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/content/${CONTENT_GUID}" | jq -r .content_url)

You can visit the CONTENT_URL in a browser, or use it to access content such as Plumber or Flask APIs. For example, if you have deployed a Plumber API that responds to POST requests on its /evaluate endpoint:

curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONTENT_URL}evaluate"

Deploying a New Version

You can deploy a new version of your content using the methods described above:

• Upload a bundle
• Deploy the bundle
• Monitor the deployment task

Performing those tasks for a second time requires that your API client provide the content_guid returned when the content was first created.

Identifying Content by Name

Clients that cannot store the content GUID can provide a unique name to identify the content item to update.

When creating your content, specify a unique value in the name field:

export NAME="my-unique-app-name"
export DATA='{"name":"'${NAME}'"}'

curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/content"

When deploying an update, you can get the content GUID using the name. Content names are guaranteed to be unique within a single user’s account, but it is possible for multiple users to create content with the same name. To ensure that a single content item is returned, provide the GUID of the user account that created the content.

# Get the GUID of the content creator (owner of this API key)
curl --silent --show-error -L --max-redirs 0 --fail -X GET \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/user"
# => {
# =>   "email": "user@example.com",
# =>   "username": "user",
# =>   ...
# =>   "guid": "0b609163-aad5-4bfd-a723-444e446344e3",
# => }

OWNER_GUID="0b609163-aad5-4bfd-a723-444e446344e3"

# Look up the app by name and owner
curl --silent --show-error -L --max-redirs 0 --fail -X GET \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/content?name=${NAME}&owner_guid=${OWNER_GUID}"
# => [
# =>   {
# =>     "guid": "ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5",
# =>     "name": "my-unique-app-name",
# =>     ...
# =>     "owner_guid": "0b609163-aad5-4bfd-a723-444e446344e3",
# =>   }
# => ]

CONTENT_GUID="ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5"

Then follow the usual update steps, passing in the CONTENT_GUID when uploading and deploying the bundle.