Overview
BOSH is a powerful tools in provisioning and deploying software over VMs consistently. BOSH will perform monitoring, failure recovery and software updates with zero-to-minimal downtime. With BOSH, application platform like VMware Tanzu Application Service (formerly Pivotal Platform Application Service) and Pivotal Container Service (PKS) can be deployed in a consistent manner.
However, sometimes there might be custom OS configuration required. BOSH os-conf release is developed to enable configuration of Linux OS, for example:
- Customize login banner text
- Add UNIX users to VM
- Add system wide CA Certificate
- Configure resolv.conf search domain
- Change TCP keepalive kernel args
- Apply arbitrary sysctls
In this example, we will create a custom SSH login user for Master and Worker nodes in BOSH deployed PKS cluster. There are various use case that can utilize os-conf BOSH release including creating temporary service account for vulnerable scanning (VA scanning) or anti-virus agent and applying custom login banner to OS.
Steps to Enable Custom SSH Login User
Steps for creating a custom SSH login for BOSH-deployed VMs is defined as below:-
- Download os-conf BOSH release into jumphost (Pivotal Operation Manager in this guide)
- Create os-conf BOSH release tarball
- Upload BOSH release tarball to BOSH director
- Create SSH key pair for VMs access
- Create runtime-config that fixed to BOSH deployment
- Run update runtime-config on BOSH
Download os-conf BOSH release
Release and source code for os-conf BOSH release can be obtained at cloudfoundry/os-conf-release. First, in jumphost (Pivotal Operation Manager in this example), we will clone os-conf source code into OpsMan directory and navigate to the directory of os-conf-release.
git clone https://github.com/cloudfoundry/os-conf-release.git
cd os-conf-release
Create os-conf BOSH release tarball
Next, we will package and create the os-conf BOSH release tarball in jumphost based on the required version of release (version 21.0.0 in this example).
While creating BOSH release, some additional packages is needed to be downloaded from internet. Do run the bosh create-release
command on jumphost that has internet access. Result of the command line will be os-conf release tarball created in .tgz format.
bosh create-release releases/os-conf/os-conf-<VERSION>.yml --tarball os-conf-<VERSION>.tgz
![](https://www.pivotal.my/wp-content/uploads/2020/04/Create-release-1024x428.png)
bosh create-release
commandUpload Tarball Release to BOSH Director
Tarball for os-conf release created in previous step will then be uploaded to BOSH director using bosh upload-release
command. Wait for BOSH director to extract, verify and create new release. Verify that os-conf release has been uploaded to BOSH director using bosh releases
command. Ensure the correct version uploaded to BOSH director.
bosh -e <BOSH_ENV> upload-release os-conf-<VERSION>.tgz
bosh -e <BOSH_ENV> releases
![](https://www.pivotal.my/wp-content/uploads/2020/04/Upload-Release.png)
bosh upload-release
command![](https://www.pivotal.my/wp-content/uploads/2020/04/Check-Release.png)
bosh releases
commandGenerate SSH Key Pair
Adding OS user to BOSH-deployed VMs required credentials to be created in the first place. In os-conf BOSH release, credentials can be added to user is two ways:-
- Username with Password
- Username with SSH private and public key pair
In this example, we will be adding user to BOSH-deployed VMs using username with SSH private and public key pair which is a more secure way of passing OS credentials. We will create the SSH private and public key pair in this step by using ssh-keygen
command. Note that other method of generating SSH key like PuTTYgen
will also work.
ssh-keygen
![](https://www.pivotal.my/wp-content/uploads/2020/04/Generate-SSH-Key-pair-1024x357.png)
ssh-keygen
command![](https://www.pivotal.my/wp-content/uploads/2020/04/Verify-SSH-Key-pair-1.png)
Create Runtime Config YAML File
Runtime config is a way that BOSH used to update or define IaaS agnostic configuration that applies to deployments. Based on created runtime config YAML file, BOSH director will apply the changes to specified deployment during the next bosh deploy
cycle.
Addons block in runtime config can be used to specify placement rules for required BOSH-deployed VMs. Placement rules is defined using include
(inclusion placement rules) or exclude
(exclusion placement rules). Available rules for the addons directives are:
- stemcell
- os – Based on stemcell’s operating system
- deployments – Based on deployment names
- jobs
- name – Based on job names
- release – Based on job release names
- instance_groups – Based on instance group names
- lifecycle – Based on lifecycle type, either
service
orerrand
- networks – Based on network names
- teams – Based on team names
Reference: Placement rules for BOSH runtime-config
releases:
- name: os-conf # Release name
version: 21.0.0 # Release version
addons:
- name: os-conf
jobs:
- name: user_add
release: os-conf
properties:
persistent_homes: true # If set to true, user home directories are created on persistent storage in /var/vcap/store/home
users:
- name: va-scan
public_key: "<PUBLIC_SSH_KEY>" # Add in Public SSH key created
shell: /bin/bash # OPTIONAL: Defaults to `/bin/bash`
sudo: true # OPTIONAL: Defaults to `true`
include:
deployments: [<DEPLOYMENT_ID>] # Add in Deployment ID obtain using bosh deployments command
Update Runtime Config in BOSH Director
After compiling of the runtime config YAML file with parameters suited to use case (applied to one PKS cluster deployment in this example), run bosh update-runtime-config
command to apply runtime config to BOSH director. Run bosh configs
to verify if runtime config updated in BOSH director.
bosh -e <BOSH_ENV> update-runtime-config --name=create_ssh_user ./runtime-config.yml
bosh -e <BOSH_ENV> configs
![](https://www.pivotal.my/wp-content/uploads/2020/04/Update-runtime-config.png)
bosh update-runtime-config
command![](https://www.pivotal.my/wp-content/uploads/2020/04/Verify-bosh-configs-1024x224.png)
bosh configs
commandOnce verified runtime config updated in BOSH director, apply changes to deployment by running apply changes
in Pivotal Operation Manager (OpsMan) or trigger update of PKS cluster using pks update-cluster
command. On OpsMan, run bosh tasks
to check status of rolling update on deployment and run bosh vms
to check on status of deployment once rolling update of VMs completed by BOSH director.
pks login -a <PKS_API_FQDN> -u <USERNAME> -p <PASSWORD>
pks update-cluster <CLUSTER_NAME> --num-nodes <NUMBER_OF_NODES>
bosh -e <BOSH_ENV> task <TASK_ID>
bosh -e <BOSH_ENV> -d <DEPLOYMENY_ID> vms
After rolling update on BOSH-deployed VMs (which are PKS nodes in this example) completed, BOSH-deployed VMs can be access using SSH username and SSH private key created in previous step.
ssh -i <PRIVATE_KEY_FILE_NAME> <USER_NAME>@<VM_IP>
![](https://www.pivotal.my/wp-content/uploads/2020/04/SSH-Login.png)
Conclusion
With this, we have successfully apply os-conf BOSH release to the deployment. External user will now be able to SSH access to BOSH-deployed VMs using SSH private key and/or password created and defined.
Reference
- BOSH os-conf Release Usage – https://bosh.io/releases/github.com/cloudfoundry/os-conf-release?version=22.0.0
- BOSH os-conf GitHub Link –https://github.com/cloudfoundry/os-conf-release/tree/ef7220d/
- BOSH Director Runtime Config Usage – https://bosh.io/docs/runtime-config/
- Generating SSH Key using SSH Keygen – https://www.ssh.com/ssh/keygen/