Skip to content

Latest commit

 

History

History
170 lines (116 loc) · 10.5 KB

README.dev.md

File metadata and controls

170 lines (116 loc) · 10.5 KB

Sourcegraph Developer README

Cutting a release

Make the desired changes to this repository

Updating docker image tags

  • The vast majority of the time, Renovate will open PRs in a timely manner.

  • If you want to update them manually, you can update the Docker image versions in *.Deployment.yaml to match the tagged version that you are releasing.

Open a PR

Wait for buildkite to pass and for your changes to be approved, then merge and check out master.

Smoke test

Test what is currently checked in to master by installing Sourcegraph on a fresh cluster.

Provision a cluster and install Sourcegraph

Using the sourcegraph/deploy-k8s-helper tool

Clone https://github.com/sourcegraph/deploy-k8s-helper to your machine and follow the README to set up all the prerequisistes.

Do smoke tests for master branch
  1. Ensure that the deploySourcegraphRoot value in your stack configuration (see https://github.com/sourcegraph/deploy-k8s-helper/blob/master/README.md) is pointing to your deploy-sourcegraph checkout (ex: pulumi config set deploySourcegraphRoot /Users/ggilmore/dev/go/src/github.com/sourcegraph/deploy-sourcegraph)
  2. In your deploy-sourcegraph checkout, make sure that you're on the latest master
  3. Run yarn up in your https://github.com/sourcegraph/deploy-k8s-helper checkout
  4. It'll take a few minutes for the cluster to be provisioned and for sourcegraph to be installed. Pulumi will show you the progresss that it's making, and will tell you when it's done.
  5. Use the instructions in configure.md to:
    1. Add a repository (e.g. sourcegraph/sourcegraph)
    2. Enable a language extension (e.g. Go), and test that code intelligence is working on the above repository
    3. Do a few test searches
  6. When you're done, run yarn destroy to tear the cluster down. This can take ~10 minutes.
Check the upgrade path from the previous release to master
  1. In your deploy-sourcegraph checkout, checkout the commit that contains the configuration for the previous release (e.g. the commit that has 2.11.x images if you're currently trying to release 2.12.x, etc.)
  2. Run yarn up in your https://github.com/sourcegraph/deploy-k8s-helper checkout
  3. Do the same smoke tests that you did above
  4. In your deploy-sourcegraph checkout, checkout the latest master commit again and run yarn up to deploy the new images. Check to see that the same smoke tests pass after the upgrade process.
  5. When you're done, run yarn destroy to tear the cluster down.
Manual instructions
Provision a new cluster
  1. Create a cluster unique name that is identifiable to you (e.g ggilmore-test) in the Sourcegraph Auxiliary GCP Project.
    • You can create a pool that has 4 nodes, each with 8 vCPUs and 30 GB memory (for a total of 32 vCPUs and 120 GB memory).
    • See this screenshot, but note that the UI could have changed:
  2. It’ll take a few minutes for it to be provisioned. You’ll see a green checkmark when it is done.
  3. Click on the connect button next to your cluster, it’ll give you a command to copy+paste in your terminal.
  4. Run the command in your terminal. Once it finishes, run kubectl config current-context. It should tell you that it’s set up to talk to the cluster that you just created.
Do smoke tests for master branch
  1. Deploy the latest master to your new cluster by running through the quickstart steps in docs/install.md
    • You'll need to create a GCP Storage Class named sourcegraph with the same zone that you created your cluster in (see "Configure a storage class")
    • In order to give yourself permissions to create roles on the cluster, run: kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user $YOUR_NAME@sourcegraph.com
  2. Use the instructions in configure.md to:
    1. Add a repository (e.g. sourcegraph/sourcegraph)
    2. Enable a language extension (e.g. Go), and test that code intelligence is working on the above repository
    3. Do a couple test searches
Check the upgrade path from the previous release to master
  1. Tear down the cluster that you created above by deleting it through from the Sourcegraph Auxiliary GCP Project.
  2. Checkout the commit that contains the configuration for the previous release (e.g. the commit has 2.11.x images if you're currently trying to release 2.12.x, etc.)
  3. Use the "Provision a new cluster" instructions above to create a new cluster.
  4. Deploy the older commit to the new cluster, and do the same smoke tests with the older version.
  5. Checkout the latest master, deploy the newer images to the same cluster (without tearing it down in between) by running ./kubectl-apply-all.sh, and check to see that the smoke test passes after the upgrade process.

Tag the release

The version numbers for sourcegraph/deploy-sourcegraph largely follow sourcegraph/sourcegraph's version numbers (i.e. deploy-sourcegraph@v2.11.2 uses sourcegraph/sourcegraph's v2.11.2 image tags).

sourcegraph/deploy-sourcegraph's branching strategy

Cutting a new minor version (e.g. v2.12.0)

  • Make a branch named 2.12 that stems from the current master branch and push it. 2.12 will be used as the base for all v2.12.x images, and future changes to the v2.12.x series will be cherry-picked onto it and tagged from there.

    git checkout master
git pull # make sure that you're up to date
git checkout -b 2.12
git push --set-upstream origin 2.12

  • Follow the tagging instructions below to tag the v2.12.0 release from the 2.12 branch.

Cutting a new patch version (e.g. v2.12.1)

  • Cherry-pick the relevant commits from master that update the image tags to v2.12.1 onto the 2.12 branch (which should have been created as mentioned above).

    git checkout 2.12
git pull # make sure that you're up to date
git cherry-pick commit0 commit1 ... # all relevant commits from the master branch
git push

  • Follow the tagging instructions below to tag the v2.12.1 release from the 2.12 branch.

Only updating Kubernetes manifests/docs

See this commit as an example of a change that would fall into this category.

  • Cherry-pick the relevant commits from master that update the manifests/docs onto the 2.12 branch.

    git checkout 2.12
git pull # make sure that you're up to date
git cherry-pick commit0 commit1 ... # all relevant commits from the master branch
git push

  • We mark these kinds of releases with a version number that looks like v2.12.0-2. Note the -2 suffix, which increments with each update like this that affects the v2.12.0 series. Look at the releases page, and pick the suffix that's next in the series.

  • Follow the tagging instructions below to tag the v2.12.0-2 release from the 2.12 branch.

Create a git tag and push it to the repository

VERSION = vX.Y.Z

# If this is a release candidate: VERSION = `vX.Y.Z-pre${N}` (where `N` starts at 0 and increments as you test/cut new versions)

# 🚨 Make sure that you have the commit that you want to tag as $VERSION checked out!

git tag $VERSION
git push origin $VERSION

Update the latestReleaseKubernetesBuild value in sourcegraph/sourcegraph

See https://sourcegraph.sgdev.org/github.com/sourcegraph/sourcegraph/-/blob/cmd/frontend/internal/app/pkg/updatecheck/handler.go#L29:5

Minikube

You can use minikube to run Sourcegraph Cluster on your development machine. However, due to minikube requirements and reduced available resources we need to modify the resources to remove resources requests/limits and storageClassNames. Here is the shell commands you can use to spin up minikube:

find base -name '*Deployment.yaml' | while read i; do yj < $i | jq 'walk(if type == "object" then del(.resources) else . end)' | jy -o $i; done
find base -name '*PersistentVolumeClaim.yaml' | while read i; do yj < $i | jq 'del(.spec.storageClassName)' | jy -o $i; done
find base -name '*StatefulSet.yaml' | while read i; do yj < $i | jq 'del(.spec.volumeClaimTemplates[] | .spec.storageClassName) | del(.spec.template.spec.containers[] | .resources)' | jy -o $i; done
minikube start
kubectl create ns src
kubens src
./kubectl-apply-all.sh
kubectl expose deployment sourcegraph-frontend --type=NodePort --name sourcegraph
kubectl expose deployment management-console --type=NodePort --name admin
minikube service list

Additionally you may want to deploy a modified version of a service locally. Minikube allows us to directly connect to its docker instance, making it easy to use unpublished images from the sourcegraph repository:

eval $(minikube docker-env)
IMAGE=repo-updater:dev ./cmd/repo-updater/build.sh
kubectl edit deployment/repo-updater # set imagePullPolicy to Never
kubectl set image deployment repo-updater '*=repo-updater:dev'