Content Execution Environments
In Posit Connect, an execution environment refers to the metadata that defines the characteristics of a Docker image. Once an execution environment has been selected, Connect uses the Docker image associated with that environment to build and execute your content on Kubernetes. Execution environments in Connect should not be confused with the Vars tab on the Content Settings panel, which is used to define environment variables. While environment variables are content-specific, execution environments are global, meaning they can be shared across content items. This appendix explains how execution environments are defined, and how they are selected by Connect to build and execute your content.
When content is first deployed, Connect uses the selected environment to create a build container, which is used to install R and Python packages required by your content. Each time your content needs to run after the initial build, that same environment is used to create an “execution container” for content execution.
Execution environments overview
Users can interact with execution environments in Connect by visiting the Environments tab in the dashboard or by interacting with the Connect server API. The environments dashboard lists all of the execution environments that are available for building and executing content on Connect. Users with the Administrator role are allowed to create, update, or remove environments. Publishers are only allowed to view available environments created by an Administrator, while Viewers cannot see the environments dashboard at all.
Execution environments describe the capabilities of a given Docker image. The capabilities of an image refer to the available Python, Quarto, and R installations for the image, and the version of each installation. Connect uses the capabilities of the image when determining whether an environment is compatible for building and executing content. The process used by Connect to select a compatible environment is discussed elsewhere.
There is a one-to-one mapping between an execution environment in Connect and the name of a Docker image. For example, the Docker image named rstudio/content-base:r4.1.0-py3.9.2-jammy can only be referenced by a single environment. This is because the shared Python and R package caches are tied to the Docker image name. Therefore, content which shares an execution environment will also share a package cache.
It is possible to create multiple environments that reference the same underlying Docker image by using
docker tagto create an alias. Images, and by extension execution environments, do not share their Python/R package caches.
The environments dashboard can be used to search for execution environments based on the environment’s capabilities. The search bar can be used to filter the list of environments based on the title, description, or installation versions. The column headers can also be sorted to customize the order of the results.
To add a new execution environment, an Administrator must enter the capabilities that describe the image. This information includes the available Python, Quarto, and R installations and their versions. The Administrator can also choose to include a human-readable title and description for the environment. This is useful for Publishers later when selecting an execution environment for their content. Optionally, the Administrator can also configure a supervisor and a matching strategy for the environment.
To add an installation, the Administrator must enter the semantic version of the interpreter and the absolute path to the interpreter’s executable.
Environment supervisors
Under the local execution model, it is possible to configure Posit Connect to invoke a program supervisor before building or executing content. The supervisor is a script or program that must be available and executable on the Connect server.
Under the off-host execution model, you can still use a supervisor, but it is necessary to configure it per-environment rather than globally (as the same script or program might not be installed on every image).
Please see the Program Supervisors section of the Posit Connect Admin Guide to read more about the uses and requirements of supervisors.
It is still possible to configure a global supervisor in the Connect configuration file when using off-host execution; it is only used when invoking processes local to the Connect server (like git processes). A warning is printed at startup to remind you that the supervisor will not be invoked when building or executing content.
Unlike the global supervisor, per-environment supervisors cannot be validated at startup. Please ensure that any supervisor you configure is available and executable in the relevant image and that it conforms to the rules described in the Admin Guide.
The configured supervisor script or program must exist and be executable within the image you are describing. One way to accomplish this is to add the supervisor script to the image during the Docker build and bake it into the image.
Matching strategy
An environment’s matching strategy tells Connect if/how the environment is available for selection. The default value, any, means that Connect can consider the environment for selection if one is not explicitly requested by the content. The value exact indicates that the environment should never be selected automatically by Connect, and it can only be used if it is requested by the content. See the section on Publisher environment selection for details about how to request a specific environment for a given piece of content. The value none means that the environment is disabled and cannot be used.
Bootstrapping execution environments
If Connect was installed using the official Helm Chart, then a default set of execution environments is created by default the first time the server starts. If any execution environments already exist in Connect, then no environments are created or modified on subsequent restarts. This allows users to bootstrap new installations by providing a set of execution environments which are created automatically the first time the server starts. To customize the set of environments that are created automatically, server administrators can modify the YAML configuration file specified by the Launcher.ClusterDefinitions setting.
This can be accomplished by modifying the values.yaml for your installation. If used for a new installation, the following values.yaml creates a single execution environment with three installations (one Python, one R, and one Quarto) with the image my-org/content-base:r4.1-py3.10.
launcher:
customRuntimeYaml:
name: "Kubernetes" # (required) this is always "Kubernetes"
images:
- name: my-org/content-base:r4.1-py3.10 # (required) a valid Docker image name
matching: any # (optional) one of [none, any, exact] defines the image selection
# behavior for this environment
supervisor: /scripts/connect-supervisor.sh # (optional) defines a supervisor script for this environment
python: # (optional) defines a set of python installations
installations:
- path: /opt/python/3.10.4/bin/python3 # (required) the absolute path to the executable
version: 3.10.4 # (required) semantic version of the installation
r: # (optional) defines a set of R installations
installations:
- path: /opt/R/4.1.3/bin/R
version: 4.1.3
quarto: # (optional) defines a set of Quarto installations
installations:
- path: /opt/quarto/bin/quarto
version: 1.0.35The YAML file is only used when no environments are defined. See the Admin dashboard, the Server API Reference, and the API Cookbook for ongoing updates.
Generating Launcher.ClusterDefinitions YAML
The build-image-yaml.sh and examine-image.sh scripts can help generate YAML for your images. These scripts are available from the content/scripts subdirectory of the rstudio/rstudio-docker-products repository.
These scripts:
Use
docker runto inspect a named image. Images not already in the local Docker cache are downloaded. Images present in the local cache are not re-pulled.Look for R installations at
/opt/R/<VERSION>and/opt/local/R/<VERSION>. If none are found, they look forRusing thePATH.Look for Python installations at
/opt/python/<VERSION>and/opt/local/python/<VERSION>. If none are found, they look forpython,python2, andpython3using thePATH.
The following command produces YAML describing three images.
./scripts/build-image-yaml.sh \
rstudio/content-base:r3.6.3-py3.8.8-jammy \
rstudio/content-base:r4.0.5-py3.8.8-jammy \
rstudio/content-base:r4.1.0-py3.9.2-jammy \
> runtimes.yamlReview the generated YAML to confirm that the expected R and Python installations are included. Remove any detected installations that you do not want in use.
Environment selection
When your content is deployed, Connect can automatically determine which environment is most appropriate for your content. Given the R and Python version demands of your content, Connect uses the configured R and Python matching algorithms to discard images that do not have compatible R and Python installations. From the remaining images, the best candidate is chosen by performing a pairwise comparison of the R and Python version components.
The Server.RVersionMatching setting controls which R installations are considered compatible with your content. We recommend a setting of major-minor rather than the default nearest configuration. The Connect Admin Guide contains more information about R version matching.
The Python.VersionMatching setting controls which Python installations are considered compatible with your content. The default of major-minor is appropriate for most installations. The Connect Admin Guide contains additional details about Python version matching.
Once an environment and appropriate R and Python installations are selected, Connect uses that environment to install R packages and Python packages before using that same environment to run your content.
Publisher selection
Publishers can select a specific environment for their content through the content settings in the dashboard, or the bundle’s manifest.json file.
Dashboard selection
To select an environment through the dashboard, navigate to the content settings. Under the Access tab, the Execution Environment section allows Publishers to select an environment to use the next time their content builds and executes. Publishers can choose between Automatic and Use a specific environment.
Note that modifying this selection will cause your content to rebuild the next time it runs.
If Automatic is selected, then Connect will attempt to choose an environment that meets the requirements of your content using the strategy described above. If Use a specific environment is selected, then the Publisher is presented with a window where they must select an environment.
The environment selection window provides the same filter and sort functionality as the main Environments dashboard. Publishers can use this window to select a compatible environment for their content.
The environment selected must satisfy the R and Python version requirements of your content using the same version-matching criteria listed above. If the environment is not compatible, then Connect cannot run the content.
The environment selected through the content settings serves as the default environment that will be used when building an executing the content. If an image is specified by the bundle’s manifest.json, then the manifest selection is used instead. This means that bundle re-activation continues to work, even when the manifest selects a different image than the content’s default execution environment.
Manifest selection
Publishers can optionally set the environment.image field in the content’s manifest.json to select a Docker image to be used for building and executing content. This option provides a code-first approach to deployments and allows Publishers to have fine-grained control over the image used to build and execute content. It also supports image selection with git-backed deployments.
The Python client package, rsconnect-python (1.8.0+), supports an --image parameter when deploying content. Similarly, the rsconnect (0.8.26+) R package supports an image argument so that users can specify an image when deploying content to Posit Connect.
The image selected by the manifest must also have a corresponding environment entry in Connect, and the R and Python version requirements of your content must be satisfied using the same version-matching criteria listed above. If the environment is not compatible, then Connect cannot run the content.
Push-button deployment from the IDE does not currently support image selection.
About automatic environment selection
In some cases, it is useful to exclude an image from automatic selection. For example, a highly customized content image that is only suitable for running a very specific content item might not be generally suitable to run most content. In this scenario, users can set the matching field in the runtimes.yaml configuration to control image selection.
There are three possible values accepted by the matching field:
any(default) indicates that the image can be selected automatically by Connect, or it can be explicitly targeted by a bundle’smanifest.json.noneindicates that the image is effectively disabled. It will never be considered for selection.exactindicates that the image can only be used when it is explicitly targeted by the bundle’smanifest.json. It will not be selected automatically by Connect, even if it is compatible with the content.
Building Docker images for use as execution environments
Images
You probably want a selection of images that reflect the versions of R and Python used by your organization. We recommend images which:
Cover each MAJOR.MINOR version of R.
This probably means images with R-3.6.X, R-4.0.X, R-4.1.X, R-4.2.X, and R-4.3.X, where X is the latest patch for each release.
Cover each MAJOR.MINOR version of Python.
This probably means images with Python-3.8.X, Python-3.9.X, Python-3.10.X and Python-3.11.X, where X is the latest patch for each release.
Posit has created a set of basic images that are available to use in your environment. Each image uses the Ubuntu 22.04 LTS (Jammy Jellyfish) release and includes single versions of R, Python, and Quarto, and system libraries commonly used by R and Python packages.
- The R installation is available at
/opt/R/<VERSION> - The Python installation is available at
/opt/python/<VERSION> - The Quarto installation is available at
/opt/quarto/<VERSION>
Available images are provided in the table below. New images may be added as new R, Python, and Quarto releases occur. If these images do not suit your organization, you can create and use custom images.
| Image Name | Python | R | Quarto |
|---|---|---|---|
ghcr.io/rstudio/content-base:r3.6.3-py3.8.16-jammy |
3.8.16 | 3.6.3 | 1.3.340 |
ghcr.io/rstudio/content-base:r4.0.5-py3.9.16-jammy |
3.9.16 | 4.0.5 | 1.3.340 |
ghcr.io/rstudio/content-base:r4.1.3-py3.10.11-jammy |
3.10.11 | 4.1.3 | 1.3.340 |
ghcr.io/rstudio/content-base:r4.2.2-py3.11.3-jammy |
3.11.3 | 4.2.2 | 1.3.340 |
These images are available from GitHub Container Registry: https://github.com/rstudio/rstudio-docker-products/pkgs/container/content-base
The source that builds these images is available from GitHub: https://github.com/rstudio/rstudio-docker-products/tree/main/content
Custom images
You are welcome to create your own images by extending the rstudio/content-base images or by creating new images from scratch.
The Server API Cookbook contains the recipe Content Runtimes - R and Python versions in use which can assist you in understanding the needs of your existing content.
Some general guidance when creating your own images:
Prefer smaller images where possible. Large images lead to increased launch times, as images may need to be downloaded from their hosting repository.
An image should contain at least one Python or R installation. Images without R or Python are be compatible with the runtime requirements of your content and are not used.
Images should include the system dependencies of the R and Python packages you use.
Consider having Posit Connect use different, smaller images than you use during development.
You do not need to include both R and Python in every image; choose combinations that mirror what you use during development.
You do not need to produce an image for every patch release of R and Python; the most recent patch for each MAJOR.MINOR release is usually sufficient.
For further information, see Creating Custom Images for Content Execution.
Runtime components
It’s possible that these images do not contain any runtime components of Posit Connect. When the job is launched, the container is augmented with a handful of required runtime components using a separate, auxiliary Posit Connect init-container. The use of an init-container to prepare the runtime components means that the content image does not need to be rebuilt for a particular version of Posit Connect. For more details about the init-container, see the Runtime Init-Container section.