Much ado about scripting, Linux & Eclipse: card subject to change

2019-12-19

Injecting registry content into Red Hat CodeReady Workspaces 2.0 (or Che 7.3.2+)

So it's been a few months since my last post - during that time, there have been six Che 7.x releases, and Red Hat CodeReady Workspaces 2.0 was released!

Continuing my series about Che (& Che accessories), here's how to inject new content into a running registry pod, without having to build a custom registry container. Why would you want this? 

Say you've got a great idea for a devfile or plugin and just want to test it out, but don't want to deal with building & releasing container(s). This technique lets you just update the running pod with your new content. 

NOTE: content won't persist if you roll out a new deployment, so don't do this in production. For that you'll want a custom container image.

Want to learn more about devfiles? Read this.

Adding a devfile at runtime


Procedure
To add a devfile:
  1. Check out the devfile registry sources.
    git clone https://github.com/redhat-developer/codeready-workspaces/; \
      cd codeready-workspaces/dependencies/che-devfile-registry
    
    # or for Che 7
    git clone https://github.com/eclipse/che-devfile-registry; \
      cd che-devfile-registry
    
  2. Create a devfile.yaml and meta.yaml in some local folder. This can be done from scratch or by copying from an existing devfile.
    STACK="new-stack"; \
      mkdir -p devfiles/${STACK}; cp devfiles/nodejs/* devfiles/${STACK}/
  3. If copying from an existing devfile, make changes to the devfile to suit your needs. Make sure your new devfile has a unique displayName and description.
  4. Get the name of the pod that hosts the devfile registry container. To do this, filter the component=devfile-registry label. Note: for Che 7, use kubectl instead of oc
    DEVFILE_REG_POD=$(oc get -o custom-columns=NAME:.metadata.name \
      --no-headers pod -l component=devfile-registry)
  5. Regenerate the registry’s index.json file to include your new devfile.
    cd .../che-devfile-registry; \
      "$(pwd)/build/scripts/check_mandatory_fields.sh" devfiles; \
      "$(pwd)/build/scripts/index.sh" > index.json
  6. Copy the new index.jsondevfile.yaml and meta.yaml files from your new local devfile folder to the container.
    cd .../che-devfile-registry; \
      LOCAL_FILES="$(pwd)/${STACK}/meta.yaml $(pwd)/${STACK}/devfile.yaml $(pwd)/index.json"; \
      oc exec ${DEVFILE_REG_POD} -i -t -- mkdir -p /var/www/html/devfiles/${STACK}; \
      for f in $LOCAL_FILES; do e=${f/$(pwd)\//}; echo "Upload ${f} -> /var/www/html/devfiles/${e}"
        oc cp "${f}" ${DEVFILE_REG_POD}:/var/www/html/devfiles/${e}; done
  7. If you have administrative rights on the cluster, you can browse to the running pod and log into it to check your changes have actually happened:

  8. The new devfile can now be used from the existing Che instance’s devfile registry. To discover it, go to the Che dashboard, then click the Workspaces link. From there, click Add Workspace to see the updated list of available devfiles.



Adding a plug-in at runtime


Procedure
To add a plug-in:
  1. Check out the plugin registry sources.
    git clone https://github.com/redhat-developer/codeready-workspaces/; \
      cd codeready-workspaces/dependencies/che-plugin-registry
    
    # or for Che 7
    git clone https://github.com/eclipse/che-plugin-registry; \
      cd che-plugin-registry
  2. Create a meta.yaml in some local folder. This can be done from scratch or by copying from an existing plug-in’s meta.yaml file.
    PLUGIN="v3/plugins/new-org/new-plugin/0.0.1"; \
      mkdir -p ${PLUGIN}; cp v3/plugins/che-incubator/cpptools/0.1/* ${PLUGIN}/
      echo "${PLUGIN##*/}" > ${PLUGIN}/../latest.txt
  3. If copying from an existing plug-in, make changes to the meta.yaml file to suit your needs. Make sure your new plug-in has a unique titledisplayName and description. Update the firstPublicationDate to today’s date.
  4. These fields in meta.yaml must match the path defined in PLUGIN above.
    publisher: new-org
    name: new-plugin
    version: 0.0.1
  5. Get the name of the pod that hosts the plug-in registry container. To do this, filter the component=plugin-registry label.Note: for Che 7, use kubectl instead of oc
    PLUGIN_REG_POD=$(oc get -o custom-columns=NAME:.metadata.name \
      --no-headers pod -l component=plugin-registry)
  6. Regenerate the registry’s index.json file to include your new plug-in.
    cd .../che-plugin-registry; \
      "$(pwd)/build/scripts/generate_latest_metas.sh" v3 && \
      "$(pwd)/build/scripts/check_plugins_location.sh" v3 && \
      "$(pwd)/build/scripts/set_plugin_dates.sh" v3 && \
      "$(pwd)/build/scripts/check_plugins_viewer_mandatory_fields.sh" v3 && \
      "$(pwd)/build/scripts/index.sh" v3 > v3/plugins/index.json
  7. Copy the new index.json and meta.yaml files from your new local plug-in folder to the container.
    cd .../che-plugin-registry; \
      LOCAL_FILES="$(pwd)/${PLUGIN}/meta.yaml $(pwd)/v3/plugins/index.json"; \
      oc exec ${PLUGIN_REG_POD} -i -t -- mkdir -p /var/www/html/${PLUGIN}; \
      for f in $LOCAL_FILES; do e=${f/$(pwd)\//}; echo "Upload ${f} -> /var/www/html/${e}"; \
        oc cp "${f}" ${PLUGIN_REG_POD}:/var/www/html/${e}; done
  8. If you have administrative rights on the cluster, you can browse to the running pod and log into it to check your changes have actually happened:



  9. The new plug-in can now be used from the existing Che instance’s plug-in registry. To discover it, go to the Che dashboard, then click the Workspaces link. From there, click the gear icon to configure one of your workspaces. Select the Plugins tab to see the updated list of available plug-ins.


Note: much of this content is derived from my contribution to https://www.eclipse.org/che/docs/che-7/editing-a-devfile-and-plug-in-at-runtime/ ... so it's not plagiarism, it's reuse of open source content. ;)

See anything wrong? Want to contribute? We're always on the lookout for people to improve or contribute to the Che 7 and the CodeReady Workspaces docs.


2019-07-25

Deploying Che 7 with custom images (air gap environment)

2019-07-19

Configuring Minishift for use with Eclipse Che and Red Hat CodeReady Workspaces

Number 4 in my Che series... a quick guide for using Minishift, in order to test drive Che 7 or Red Hat CodeReady Workspaces.

If you're not running Windows as your host OS, your mileage may vary in terms of memory requirements and preferred virtual machine driver, but the suggestions below should work on most OSes... even the one with an incomplete sed implementation.

Get Minishift and oc

Download and unpack the latest minishift v1.34.1 binary from here:
You’ll also want the Openshift Origin Client Tools (oc) 4 or 3.11 from:

Once downloaded and unpacked, put the minishift and oc executables in your ~/bin or some other folder that’s on your PATH.

Configure Minishift

On Windows, I use Git bash shell for most shell commands.
# To prevent rate limiting when starting minishift, use a personal 
# access token.
# You can create new one at https://github.com/settings/tokens
export MINISHIFT_GITHUB_API_TOKEN="f00cafebabel33t"

# Grant enough memory to run CRW 1.2 / Che 6 and workspaces
# 6G min for one workspace, plus 2G for each additional workspace
minishift config set memory 12G

# Grant enough disk for downloading container images
minishift config set disk-size 25G

# if not using default virtualization engine, eg., want Virtual Box, use
minishift config set vm-driver virtualbox

# enable cluster admin user
minishift addons install --defaults
minishift addons enable admin-user
On Windows, you might need to touch a file that your shell can’t find.
touch /c/Users/YOUR_LOGIN_HERE/.minishift/machines/minishift_kubeconfig
Note      
If it fails with error Checking if requested OpenShift version 'v3.11.0' is valid …​ v3.11.0 is not a valid OpenShift versionFAIL, try WSL bash.
# start minishift oc cluster
minishift start
Once minishift starts successfully, you’re ready to deploy Che or CRW to the server.

If minishift fails to come up, try killing and starting it again:
minishift stop
minishift start
Or delete it and try again:
minishift stop
minishift delete
minishift start
When successful, you should see something like this (use your actual IP address in place of 192.168.$YOUR_IP):
OpenShift server started.

The server is accessible via web console at:
    https://192.168.$YOUR_IP:8443/console
You can log into the above console URL using the special user = system:admin - full cluster admin privileges required to deploy an operator:
oc login 192.168.$YOUR_IP:8443 -u system:admin
If you prefer to use a different user, you can grant that user cluster admin privileges:
oc adm policy --as=system:admin add-cluster-role-to-user cluster-admin admin # for admin user
oc adm policy --as=system:admin add-cluster-role-to-user cluster-admin developer # for developer user

Deploy something!

If you want to install CodeReady Workspaces 1.2, download the Operator Installer from here:

Set up your authentication with the new Red Hat Container Catalog's registry, registry.redhat.io:

Run the installation as follows:

./deploy.sh -d
Or, if you'd like to install Che 7, see Deploying Che 7 to Minishift.

You can also use the built-in Che add-on in Minishift, but that approach is deprecated. Enable it as follows:

minishift addons enable che; minishift addons apply \
  --addon-env CHE_DOCKER_IMAGE=eclipse/che-server:nightly \
  --addon-env OPENSHIFT_TOKEN=$(oc whoami -t) che

# switch to Che 7.0.0.RC3 plugin registry format (fixed in minishift v1.34.1) 
# see https://github.com/minishift/minishift/commit/4c0a0c0b9cfb92798fef53e5c54e80ca71c5f715
minishift --profile che addons apply --addon-env PLUGIN__REGISTRY__URL=https://che-plugin-registry.openshift.io/v3 che

2019-07-04

Creating your own devfile registry

Third in this series... how to create and deploy your own custom devfile registry, so you can include your own stacks out of the box inside Che 7.

Creating your own devfile registry

Prerequisites

You need a Kubernetes or OpenShift cluster (eg., minikube or minishift), into which you have installed Che. Follow the steps in this document to deploy Che.
Steps below can then be done to deploy a custom devfile registry to your cluster, and use it with Che.

Procedure

  1. git clone https://github.com/eclipse/che-devfile-registry/
    cd che-devfile-registry
  2. Make changes in the devfiles/ folder to add new or remove existing devfile folders from the registry.
  3. Build the image.
    This example uses a personal quay repository and an arbitrary tag ubi8-2, but you can use another image registry and path as needed.
    docker build . -f Dockerfile --no-cache \
      -t quay.io/nickboldt/che-devfile-registry:ubi8-2
  4. Push it to quay (or your image registry of choice)
    docker push quay.io/nickboldt/che-devfile-registry:ubi8-2
    Verify the image and tag you pushed are published, eg.,
    Verify the Repository Visibility is public, eg.,
  5. Remove any existing che-devfile-registry deployments.
    oc delete dc che-devfile-registry; \
    oc delete svc che-devfile-registry; \
    oc delete route che-devfile-registry; \
    sleep 5s
  6. Deploy the new che-devfile-registry.
    oc new-app -f deploy/openshift/che-devfile-registry.yaml \
      -p PULL_POLICY="Always" \
      -p IMAGE="quay.io/nickboldt/che-devfile-registry" \
      -p IMAGE_TAG="ubi8-2"; \
    sleep 5s
     
  7. You can browse the registry’s devfiles/index.json file to verify your new devfile is included.
  8. Update your config map to point to the new registry.
    oc set env dc/che CHE_WORKSPACE_DEVFILE__REGISTRY__URL=http://che-devfile-registry-che.192.168.99.112.nip.io/
  9. Check your Che deployment to see if the changes are live by creating a new workspace from your devfile registry:

    2019-06-29

    Discovering containers loaded by Che 7

    Second in this series... more tips for Che 7 on Minishift.

    Discovering containers loaded by Che 7

    Prerequisites

    You need a Kubernetes or OpenShift cluster (eg., minikube or minishift), into which you have installed Che.
    Follow the steps in this document to deploy Che. Steps below can be done in parallel to collect a list of containers involved in deployment.

    Procedure

    1. Once you have started your cluster, you can watch events and see which images are pulled. Start this in a console window before deploying Che to your cluster.
      oc project che
      oc get po
      oc get events -w | tee /tmp/minishift.log.txt
    2. Watch for lines containing the following lines to see what containers are pulled.
      Successfully pulled image "..."
        and
      Container image "..." already present on machine
    3. For example, this should extract just a list of pulled containers, including any duplicate re-pulls:
      cat /tmp/minishift.log.txt | \
        egrep "Successfully pulled image|Container image" | \
        sed -e 's#.*\(Successfully pulled image\|Container image\) "\(.\+\)\".*#\2#g'
    4. You can also look at the available images in the minishift docker context:
      eval $(minishift docker-env)
      docker images
    5. Next, using chectldeploy Che to your cluster and watch the events get logged.
    6. You can also see container events at this URL:
      • https://192.168.YOUR.IP:8443/console/project/che/browse/events