Deploy All-in-One OpenStack with Kolla-Ansible on Ubuntu 22.04

Kifarunix
|
Last Updated:
|
OpenStack, Ansible, Automation, Cloud Compute, Containers, Docker, HowTos, Virtualization
|
Leave a Comment

In this guide, you will to learn how to deploy all-in-one OpenStack with Kolla-Ansible on Ubuntu 22.04.

Kolla provides Docker containers and Ansible playbooks to meet Kolla’s mission. Kolla’s mission is to provide production-ready containers and deployment tools for operating OpenStack clouds. It allows the operators with minimal experience to deploy OpenStack quickly and as experience grows modify the OpenStack configuration to suit the operator’s exact requirements.

The current Kolla-Ansible release as of this writing is 2023.1.

Deploy All-In-One OpenStack with Kolla-Ansible on Ubuntu 22.04

System Requirements

Below are the recommended minimum requirements for deploying AIO OpenStack with Kolla-Ansible:

  • 2 (or more) network interfaces.
  • At least 8gb main memory
  • At least 40gb disk space (We will use 150G in this guide)

Below are our deployment system specifics;

Interfaces2 network interfaces:
enp1s0: 192.168.122.100/24
enp2s0: no assigned IP address
RAM8 GB
vCPUs2
Storage/dev/vda (root filesystem), /: 50 GB
/dev/vdb (Volume group, cinder): 100 GB
Virtualization PlatformKVM
Operating SystemUbuntu 22.04 LTS
Usernon root user with passwordless sudo rights

You can provide as much resources since the more resources you have the better the performance of the stack.

NOTE: We are running the installation as non root user with sudo privileges.

We are running Kolla-ansible deployment using the kifarunix user with passwordless sudo rights;

cat /etc/sudoers.d/kifarunix
kifarunix ALL = NOPASSWD: ALL

Install Required Packages on Ubuntu 22.04

Before you can proceed, there are a number of required packages that needs to be installed.

Update and upgrade your system packages

sudo apt update
sudo apt upgrade

Reboot the system if required;

[ -f /var/run/reboot-required ] && sudo systemctl reboot

Install the required packages;

sudo apt install git python3-dev libffi-dev python3-venv gcc libssl-dev git python3-pip

Create a virtual environment for deploying Kolla-ansible

To avoid conflict between system packages and Kolla-ansible packages, it is recommended that Kolla-ansible be installed in a python virtual environment (virtualenv).

You can create a python virtual environment by executing the command below. Be sure to replace the path to your virtual environment.

python3 -m venv $HOME/kolla-openstack

Next, activate your virtual environment;

source $HOME/kolla-openstack/bin/activate

Once you activate the Kolla-ansible virtual environment, you shell prompt should change. See my shell prompt;

(kolla-openstack) kifarunix@kolla-ansible:~$

To exit the virtual environment, run;

deactivate

Upgrade Python PIP

Upgrade pip;

pip install -U pip

Install Ansible on Ubuntu 22.04

Install Ansible from the virtual environment. If you ever log out of the virtual environment, you can always source the path to activate it;

source $HOME/kolla-openstack/bin/activate

Next, install Ansible. The current release version of Kolla, which is 2023.1, as of this writing, requires Ansible 6 and supports up to 7. 

pip install 'ansible>=6,<8'

Create Ansible Configuration file

Create an Ansible configuration file on your home directory with the following tunables;

vim $HOME/ansible.cfg
[defaults]
host_key_checking=False
pipelining=True
forks=100

This defines the default settings to apply to Ansible.

  • host_key_checking=False: This setting disables host key checking for SSH connections. When host_key_checking is set to False, Ansible will not prompt for confirmation when connecting to new SSH hosts for the first time. You may want to enable this though!
  • pipelining=True: Enabling pipelining allows Ansible to execute tasks in a more efficient way. When pipelining is enabled, Ansible sends multiple commands to a target host in a single SSH session, reducing the overhead of opening and closing SSH connections for each task. This can improve playbook execution performance.
  • forks=100: This setting specifies the maximum number of parallel processes or “forks” that Ansible can use when executing tasks across multiple hosts. In this case, it’s set to 100, meaning that Ansible can run up to 100 tasks concurrently. The appropriate value for forks depends on your system’s resources and the scale of your infrastructure. It’s crucial to choose a value that balances performance and resource utilization. Setting it too high can strain your system, while setting it too low may slow down playbook execution.

Install Kolla-ansible on Ubuntu 22.04

Install Kolla-ansible, along side all required dependencies on Ubuntu 22.04 using pip from the virtual environment above;

source $HOME/kolla-openstack/bin/activate

The command below installs current stable version of Kolla-ansible, as of this writing. Be sure to update the command accordingly.

pip install git+https://opendev.org/openstack/kolla-ansible@stable/2023.1

Configure Kolla-ansible for All-in-one OpenStack Deployment

Next, create Kolla configuration directory;

sudo mkdir /etc/kolla

Update the ownership of the Kolla configuration directory to the user with which you activated Kolla-ansible deployment virtual environment as.

sudo chown $USER:$USER /etc/kolla

Copy the main Kolla configuration file, globals.yml and the OpenStack services passwords file, passwords.yml into the Kolla configuration directory above from the virtual environment.

cp $HOME/kolla-openstack/share/kolla-ansible/etc_examples/kolla/* /etc/kolla/

Copy Kolla-ansible deployment inventory to the current working directory. In this tutorial, we are deploying all-in-one OpenStack with Kolla-ansible. Hence, copy the all-in-one ansible inventory file.

cp $HOME/kolla-openstack/share/kolla-ansible/ansible/inventory/all-in-one .

Define Kolla-Ansible Global Deployment Options

Open the globals.yml configuration file and define the AIO Kolla global deployment options;

vim /etc/kolla/globals.yml

Update the configuration as per your environment setup. Some of the services we enabled are not actually necessary. Be sure to enable what you really need!

Below are the basic options that we enabled for our AIO OpenStack deployment.

grep -vE '^$|^#' /etc/kolla/globals.yml
---

###################
# Ansible options
###################

workaround_ansible_issue_8743: yes

###############
# Kolla options
###############

config_strategy: "COPY_ALWAYS"
kolla_base_distro: "ubuntu"
openstack_release: "2023.1"
kolla_internal_vip_address: "192.168.122.100"
kolla_internal_fqdn: "openstack.kifarunix.com"
kolla_external_vip_address: "{{ kolla_internal_vip_address }}"
kolla_external_fqdn: "{{ kolla_internal_fqdn }}"

################
# Container engine
################

kolla_container_engine: docker

##############################
# Neutron - Networking Options
##############################

network_interface: "enp1s0"
neutron_external_interface: "enp2s0"
neutron_plugin_agent: "openvswitch"

###################
# OpenStack options
###################

enable_glance: "{{ enable_openstack_core | bool }}"
enable_haproxy: "no"
enable_keystone: "{{ enable_openstack_core | bool }}"
enable_mariadb: "yes"
enable_memcached: "yes"
enable_neutron: "{{ enable_openstack_core | bool }}"
enable_nova: "{{ enable_openstack_core | bool }}"
enable_aodh: "yes"
enable_ceilometer: "yes"
enable_cinder: "yes"
enable_cinder_backend_lvm: "yes"
enable_gnocchi: "yes"
enable_gnocchi_statsd: "yes"
enable_grafana: "yes"
enable_grafana_external: "{{ enable_grafana | bool }}"
enable_heat: "{{ enable_openstack_core | bool }}"
enable_horizon: "{{ enable_openstack_core | bool }}"
enable_nova_ssh: "yes"
enable_prometheus: "yes"

################################
# Cinder - Block Storage Options
################################

cinder_volume_group: "cinder-volumes"

Note that we enabled cinder block storage for OpenStack and defined the name of the existing volume group.

sudo vgs
  VG        #PV #LV #SN Attr   VSize    VFree   
  cinder-volumes      1   0   0 wz--n- <100.00g <100.00g
  ubuntu-vg   1   1   0 wz--n-  <48.00g

Refer to Kolla-ansible documentation guide to learn more about the global options used above. The configuration is also highly commented. Go through the comments for each option to learn what it is about a specific option.

Generate Kolla Passwords

Kolla passwords.yml configuration file stores various OpenStack services passwords. You can automatically generate the password using the Kolla-ansible kolla-genpwd in your virtual environment.

Ensure that your virtual environment is activated

source $HOME/kolla-openstack/bin/activate

Next, generate the passwords;

kolla-genpwd

All generated passwords will be populated to /etc/kolla/passwords.yml file.

Configure All-in-one OpenStack deployment Inventory

You now have your deployment inventory in place.

Since we are running an all-in-one deployment, we will leave all the default options defined on the all-in-one inventory file as is.

cat all-in-one
# These initial groups are the only groups required to be modified. The
# additional groups are for more control of the environment.
[control]
localhost       ansible_connection=local

[network]
localhost       ansible_connection=local

[compute]
localhost       ansible_connection=local

[storage]
localhost       ansible_connection=local

[monitoring]
localhost       ansible_connection=local

[deployment]
localhost       ansible_connection=local

# You can explicitly specify which hosts run each project by updating the
# groups in the sections below. Common services are grouped together.

[common:children]
control
network
compute
storage
monitoring

[collectd:children]
compute

[baremetal:children]
control

[tls-backend:children]
control

[grafana:children]
monitoring

[etcd:children]
control

[kafka:children]
control

[telegraf:children]
compute
control
monitoring
network
storage

[hacluster:children]
control

[hacluster-remote:children]
compute

[loadbalancer:children]
network

[mariadb:children]
control

[rabbitmq:children]
control

[outward-rabbitmq:children]
control

[monasca-agent:children]
compute
control
monitoring
network
storage

[monasca:children]
monitoring

[storm:children]
monitoring

[keystone:children]
control

[glance:children]
control

[nova:children]
control

[neutron:children]
network

[openvswitch:children]
network
compute
manila-share

[cinder:children]
control

[cloudkitty:children]
control

[freezer:children]
control

[memcached:children]
control

[horizon:children]
control

[swift:children]
control

[barbican:children]
control

[heat:children]
control

[murano:children]
control

[ironic:children]
control

[influxdb:children]
monitoring

[prometheus:children]
monitoring

[magnum:children]
control

[sahara:children]
control

[solum:children]
control

[mistral:children]
control

[manila:children]
control

[gnocchi:children]
control

[ceilometer:children]
control

[aodh:children]
control

[cyborg:children]
control
compute

[tacker:children]
control

[vitrage:children]
control

[senlin:children]
control

[trove:children]
control

[watcher:children]
control

[octavia:children]
control

[designate:children]
control

[placement:children]
control

[bifrost:children]
deployment

[zookeeper:children]
control

[zun:children]
control

[skyline:children]
control

[redis:children]
control

[blazar:children]
control

[venus:children]
monitoring

# Additional control implemented here. These groups allow you to control which
# services run on which hosts at a per-service level.
#
# Word of caution: Some services are required to run on the same host to
# function appropriately. For example, neutron-metadata-agent must run on the
# same host as the l3-agent and (depending on configuration) the dhcp-agent.

# Common
[cron:children]
common

[fluentd:children]
common

[kolla-logs:children]
common

[kolla-toolbox:children]
common

[opensearch:children]
control

# Opensearch dashboards
[opensearch-dashboards:children]
opensearch

# Glance
[glance-api:children]
glance

# Nova
[nova-api:children]
nova

[nova-conductor:children]
nova

[nova-super-conductor:children]
nova

[nova-novncproxy:children]
nova

[nova-scheduler:children]
nova

[nova-spicehtml5proxy:children]
nova

[nova-compute-ironic:children]
nova

[nova-serialproxy:children]
nova

# Neutron
[neutron-server:children]
control

[neutron-dhcp-agent:children]
neutron

[neutron-l3-agent:children]
neutron

[neutron-metadata-agent:children]
neutron

[neutron-ovn-metadata-agent:children]
compute
network

[neutron-ovn-agent:children]
compute

[neutron-bgp-dragent:children]
neutron

[neutron-infoblox-ipam-agent:children]
neutron

[neutron-metering-agent:children]
neutron

[ironic-neutron-agent:children]
neutron

# Cinder
[cinder-api:children]
cinder

[cinder-backup:children]
storage

[cinder-scheduler:children]
cinder

[cinder-volume:children]
storage

# Cloudkitty
[cloudkitty-api:children]
cloudkitty

[cloudkitty-processor:children]
cloudkitty

# Freezer
[freezer-api:children]
freezer

[freezer-scheduler:children]
freezer

# iSCSI
[iscsid:children]
compute
storage
ironic

[tgtd:children]
storage

# Manila
[manila-api:children]
manila

[manila-scheduler:children]
manila

[manila-share:children]
network

[manila-data:children]
manila

# Swift
[swift-proxy-server:children]
swift

[swift-account-server:children]
storage

[swift-container-server:children]
storage

[swift-object-server:children]
storage

# Barbican
[barbican-api:children]
barbican

[barbican-keystone-listener:children]
barbican

[barbican-worker:children]
barbican

# Trove
[trove-api:children]
trove

[trove-conductor:children]
trove

[trove-taskmanager:children]
trove

# Heat
[heat-api:children]
heat

[heat-api-cfn:children]
heat

[heat-engine:children]
heat

# Murano
[murano-api:children]
murano

[murano-engine:children]
murano

# Monasca
[monasca-agent-collector:children]
monasca-agent

[monasca-agent-forwarder:children]
monasca-agent

[monasca-agent-statsd:children]
monasca-agent

[monasca-api:children]
monasca

[monasca-log-persister:children]
monasca

[monasca-log-metrics:children]
monasca

[monasca-thresh:children]
monasca

[monasca-notification:children]
monasca

[monasca-persister:children]
monasca

# Storm
[storm-worker:children]
storm

[storm-nimbus:children]
storm

# Ironic
[ironic-api:children]
ironic

[ironic-conductor:children]
ironic

[ironic-inspector:children]
ironic

[ironic-tftp:children]
ironic

[ironic-http:children]
ironic

# Magnum
[magnum-api:children]
magnum

[magnum-conductor:children]
magnum

# Solum
[solum-api:children]
solum

[solum-worker:children]
solum

[solum-deployer:children]
solum

[solum-conductor:children]
solum

[solum-application-deployment:children]
solum

[solum-image-builder:children]
solum

# Mistral
[mistral-api:children]
mistral

[mistral-executor:children]
mistral

[mistral-engine:children]
mistral

[mistral-event-engine:children]
mistral

# Aodh
[aodh-api:children]
aodh

[aodh-evaluator:children]
aodh

[aodh-listener:children]
aodh

[aodh-notifier:children]
aodh

# Cyborg
[cyborg-api:children]
cyborg

[cyborg-agent:children]
compute

[cyborg-conductor:children]
cyborg

# Gnocchi
[gnocchi-api:children]
gnocchi

[gnocchi-statsd:children]
gnocchi

[gnocchi-metricd:children]
gnocchi

# Sahara
[sahara-api:children]
sahara

[sahara-engine:children]
sahara

# Ceilometer
[ceilometer-central:children]
ceilometer

[ceilometer-notification:children]
ceilometer

[ceilometer-compute:children]
compute

[ceilometer-ipmi:children]
compute

# Multipathd
[multipathd:children]
compute
storage

# Watcher
[watcher-api:children]
watcher

[watcher-engine:children]
watcher

[watcher-applier:children]
watcher

# Senlin
[senlin-api:children]
senlin

[senlin-conductor:children]
senlin

[senlin-engine:children]
senlin

[senlin-health-manager:children]
senlin

# Octavia
[octavia-api:children]
octavia

[octavia-driver-agent:children]
octavia

[octavia-health-manager:children]
octavia

[octavia-housekeeping:children]
octavia

[octavia-worker:children]
octavia

# Designate
[designate-api:children]
designate

[designate-central:children]
designate

[designate-producer:children]
designate

[designate-mdns:children]
network

[designate-worker:children]
designate

[designate-sink:children]
designate

[designate-backend-bind9:children]
designate

# Placement
[placement-api:children]
placement

# Zun
[zun-api:children]
zun

[zun-wsproxy:children]
zun

[zun-compute:children]
compute

[zun-cni-daemon:children]
compute

# Skyline
[skyline-apiserver:children]
skyline

[skyline-console:children]
skyline

# Tacker
[tacker-server:children]
tacker

[tacker-conductor:children]
tacker

# Vitrage
[vitrage-api:children]
vitrage

[vitrage-notifier:children]
vitrage

[vitrage-graph:children]
vitrage

[vitrage-ml:children]
vitrage

[vitrage-persistor:children]
vitrage

# Blazar
[blazar-api:children]
blazar

[blazar-manager:children]
blazar

# Prometheus
[prometheus-node-exporter:children]
monitoring
control
compute
network
storage

[prometheus-mysqld-exporter:children]
mariadb

[prometheus-haproxy-exporter:children]
loadbalancer

[prometheus-memcached-exporter:children]
memcached

[prometheus-cadvisor:children]
monitoring
control
compute
network
storage

[prometheus-alertmanager:children]
monitoring

[prometheus-openstack-exporter:children]
monitoring

[prometheus-elasticsearch-exporter:children]
opensearch

[prometheus-blackbox-exporter:children]
monitoring

[prometheus-libvirt-exporter:children]
compute

[prometheus-msteams:children]
prometheus-alertmanager

[masakari-api:children]
control

[masakari-engine:children]
control

[masakari-hostmonitor:children]
control

[masakari-instancemonitor:children]
compute

[ovn-controller:children]
ovn-controller-compute
ovn-controller-network

[ovn-controller-compute:children]
compute

[ovn-controller-network:children]
network

[ovn-database:children]
control

[ovn-northd:children]
ovn-database

[ovn-nb-db:children]
ovn-database

[ovn-sb-db:children]
ovn-database

[venus-api:children]
venus

[venus-manager:children]
venus

Deploy All-In-One OpenStack with Kolla-Ansible on Ubuntu 22.04

Since everything is setup, you can now start to deploy OpenStack using Kolla-ansible playbooks.

Again, ensure that your virtual environment is activated.

source $HOME/kolla-openstack/bin/activate

Install Ansible Galaxy requirements

The Kolla Ansible Galaxy requirements are a set of Ansible roles and collections that are required to deploy OpenStack using Kolla Ansible.

To install them, run the command below;

kolla-ansible install-deps

Bootstrap Kolla-Ansible Nodes

Bootstrap your localhost configuration before deploying containers using bootstrap-servers sub-command.

This is what the bootstrap command do;

  • Customization of /etc/hosts
  • Creation of user and group
  • Kolla configuration directory
  • Package installation and removal
  • Docker engine installation and configuration
  • Disabling firewalls
  • Creation of Python virtual environment
  • Configuration of Apparmor
  • Configuration of NTP daemon
  • e.t.c
kolla-ansible -i all-in-one bootstrap-servers

Below is a sample output of the bootstrapping command;

Bootstrapping servers : ansible-playbook -e @/etc/kolla/globals.yml  -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla  -e kolla_action=bootstrap-servers /home/kifarunix/kolla-openstack/share/kolla-ansible/ansible/kolla-host.yml  --inventory all-in-one
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

PLAY [Gather facts for all hosts] **************************************************************************************************************************************************

TASK [Gather facts] ****************************************************************************************************************************************************************
ok: [localhost]

TASK [Gather package facts] ********************************************************************************************************************************************************
skipping: [localhost]

TASK [Group hosts to determine when using --limit] *********************************************************************************************************************************
ok: [localhost]
[WARNING]: Could not match supplied host pattern, ignoring: all_using_limit_True

PLAY [Gather facts for all hosts (if using --limit)] *******************************************************************************************************************************
skipping: no hosts matched

PLAY [Apply role baremetal] ********************************************************************************************************************************************************

TASK [openstack.kolla.etc_hosts : Include etc-hosts.yml] ***************************************************************************************************************************
included: /home/kifarunix/.ansible/collections/ansible_collections/openstack/kolla/roles/etc_hosts/tasks/etc-hosts.yml for localhost

TASK [openstack.kolla.etc_hosts : Ensure localhost in /etc/hosts] ******************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.etc_hosts : Ensure hostname does not point to 127.0.1.1 in /etc/hosts] ***************************************************************************************
ok: [localhost]

TASK [openstack.kolla.etc_hosts : Generate /etc/hosts for all of the nodes] ********************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.etc_hosts : Check whether /etc/cloud/cloud.cfg exists] *******************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.etc_hosts : Disable cloud-init manage_etc_hosts] *************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.baremetal : Ensure unprivileged users can use ping] **********************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.baremetal : Set firewall default policy] *********************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.baremetal : Check if firewalld is installed] *****************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.baremetal : Disable firewalld] *******************************************************************************************************************************
skipping: [localhost] => (item=firewalld) 
skipping: [localhost]

TASK [openstack.kolla.packages : Install packages] *********************************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.packages : Remove packages] **********************************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : include_tasks] **************************************************************************************************************************************
included: /home/kifarunix/.ansible/collections/ansible_collections/openstack/kolla/roles/docker/tasks/repo-Debian.yml for localhost

TASK [openstack.kolla.docker : Install CA certificates and gnupg packages] *********************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Ensure apt sources list directory exists] ***********************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Ensure apt keyrings directory exists] ***************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Install docker apt gpg key] *************************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Install docker apt pin] *****************************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.docker : Enable docker apt repository] ***********************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Check which containers are running] *****************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Check if docker systemd unit exists] ****************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Mask the docker systemd unit on Debian/Ubuntu] ******************************************************************************************************
changed: [localhost]

TASK [openstack.kolla.docker : Install packages] ***********************************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Start docker] ***************************************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.docker : Wait for Docker to start] ***************************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.docker : Ensure containers are running after Docker upgrade] *************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.docker : Ensure docker config directory exists] **************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Write docker config] ********************************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Remove old docker options file] *********************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker : Ensure docker service directory exists] *************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.docker : Configure docker service] ***************************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.docker : Ensure the path for CA file for private registry exists] ********************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.docker : Ensure the CA file for private registry exists] *****************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.docker : Flush handlers] *************************************************************************************************************************************

TASK [openstack.kolla.docker : Start and enable docker] ****************************************************************************************************************************
changed: [localhost]

TASK [openstack.kolla.docker : include_tasks] **************************************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.kolla_user : Ensure groups are present] **********************************************************************************************************************
skipping: [localhost] => (item=docker) 
skipping: [localhost] => (item=sudo) 
skipping: [localhost] => (item=kolla) 
skipping: [localhost]

TASK [openstack.kolla.kolla_user : Create kolla user] ******************************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.kolla_user : Add public key to kolla user authorized keys] ***************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.kolla_user : Grant kolla user passwordless sudo] *************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.docker_sdk : Install packages] *******************************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.docker_sdk : Install latest pip in the virtualenv] ***********************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.docker_sdk : Install docker SDK for python] ******************************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.baremetal : Ensure node_config_directory directory exists] ***************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.apparmor_libvirt : include_tasks] ****************************************************************************************************************************
included: /home/kifarunix/.ansible/collections/ansible_collections/openstack/kolla/roles/apparmor_libvirt/tasks/remove-profile.yml for localhost

TASK [openstack.kolla.apparmor_libvirt : Get stat of libvirtd apparmor profile] ****************************************************************************************************
ok: [localhost]

TASK [openstack.kolla.apparmor_libvirt : Get stat of libvirtd apparmor disable profile] ********************************************************************************************
ok: [localhost]

TASK [openstack.kolla.apparmor_libvirt : Remove apparmor profile for libvirt] ******************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.baremetal : Change state of selinux] *************************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.baremetal : Set https proxy for git] *************************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.baremetal : Set http proxy for git] **************************************************************************************************************************
skipping: [localhost]

TASK [openstack.kolla.baremetal : Configure ceph for zun] **************************************************************************************************************************
skipping: [localhost]

PLAY RECAP *************************************************************************************************************************************************************************
localhost                  : ok=31   changed=2    unreachable=0    failed=0    skipped=23   rescued=0    ignored=0

Deploy AIO OpenStack with Kolla-Ansible

Run pre-deployment checks for host;

kolla-ansible -i all-in-one prechecks

If everything is fine, proceed to deploy all-in-one OpenStack with Kolla-ansible on Ubuntu 22.04;

kolla-ansible -i all-in-one deploy

The process might take a while as it involves building containers for different OpenStack services.

If all ends well, you should get 0 failed tasks;

...
PLAY RECAP *************************************************************************************************************************************************************************
localhost                  : ok=421  changed=293  unreachable=0    failed=0    skipped=189  rescued=0    ignored=1

All-in-one OpenStack Post Deployment Tasks

Add Kolla-Ansible Deployment User to Docker Group

You can optionally add your Kolla-ansible deployment user to Docker group to as to manage Docker without necessarily using sudo;

sudo usermod -aG docker $USER

To activate the new group membership, the user generally needs to log out and then log back in. This is because group memberships are set during the user's login session;

So just press ctrl+d or just type exit on the terminal to exit and re-login.

List Running OpenStack Docker Containers

Once the deployment is done, you can list running OpenStack docker containers.

docker ps
CONTAINER ID   IMAGE                                                                       COMMAND                  CREATED             STATUS                         PORTS     NAMES
8af7959f36ee   quay.io/openstack.kolla/grafana:2023.1-ubuntu-jammy                         "dumb-init --single-…"   About an hour ago   Up About an hour                         grafana
3a7876c0d974   quay.io/openstack.kolla/aodh-notifier:2023.1-ubuntu-jammy                   "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               aodh_notifier
162f6e34f66b   quay.io/openstack.kolla/aodh-listener:2023.1-ubuntu-jammy                   "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               aodh_listener
d270d1833805   quay.io/openstack.kolla/aodh-evaluator:2023.1-ubuntu-jammy                  "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               aodh_evaluator
700e2b4153e5   quay.io/openstack.kolla/aodh-api:2023.1-ubuntu-jammy                        "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               aodh_api
2b18506192e7   quay.io/openstack.kolla/ceilometer-compute:2023.1-ubuntu-jammy              "dumb-init --single-…"   About an hour ago   Up About an hour (unhealthy)             ceilometer_compute
6a6c75cde198   quay.io/openstack.kolla/ceilometer-central:2023.1-ubuntu-jammy              "dumb-init --single-…"   About an hour ago   Up About an hour (unhealthy)             ceilometer_central
cff98bd145f4   quay.io/openstack.kolla/ceilometer-notification:2023.1-ubuntu-jammy         "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               ceilometer_notification
a728899cf252   quay.io/openstack.kolla/gnocchi-statsd:2023.1-ubuntu-jammy                  "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               gnocchi_statsd
d4b0cff90c9d   quay.io/openstack.kolla/gnocchi-metricd:2023.1-ubuntu-jammy                 "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               gnocchi_metricd
4a75faa8f3e9   quay.io/openstack.kolla/gnocchi-api:2023.1-ubuntu-jammy                     "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               gnocchi_api
fdff70efac7f   quay.io/openstack.kolla/horizon:2023.1-ubuntu-jammy                         "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               horizon
2b18ecf5cbd7   quay.io/openstack.kolla/heat-engine:2023.1-ubuntu-jammy                     "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               heat_engine
cff2efa249a2   quay.io/openstack.kolla/heat-api-cfn:2023.1-ubuntu-jammy                    "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               heat_api_cfn
e764dd422bbd   quay.io/openstack.kolla/heat-api:2023.1-ubuntu-jammy                        "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               heat_api
2cf52a70a457   quay.io/openstack.kolla/neutron-metadata-agent:2023.1-ubuntu-jammy          "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               neutron_metadata_agent
4d81a8ff572f   quay.io/openstack.kolla/neutron-l3-agent:2023.1-ubuntu-jammy                "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               neutron_l3_agent
1bfaeb97f62c   quay.io/openstack.kolla/neutron-dhcp-agent:2023.1-ubuntu-jammy              "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               neutron_dhcp_agent
7269474b24f6   quay.io/openstack.kolla/neutron-openvswitch-agent:2023.1-ubuntu-jammy       "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               neutron_openvswitch_agent
c7cb525234dd   quay.io/openstack.kolla/neutron-server:2023.1-ubuntu-jammy                  "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               neutron_server
effadab3fab0   quay.io/openstack.kolla/openvswitch-vswitchd:2023.1-ubuntu-jammy            "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               openvswitch_vswitchd
edad0bc41b8b   quay.io/openstack.kolla/openvswitch-db-server:2023.1-ubuntu-jammy           "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               openvswitch_db
6a0f1c488fd0   quay.io/openstack.kolla/nova-compute:2023.1-ubuntu-jammy                    "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               nova_compute
622d2b4c2af0   quay.io/openstack.kolla/nova-libvirt:2023.1-ubuntu-jammy                    "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               nova_libvirt
364a2f8afa02   quay.io/openstack.kolla/nova-ssh:2023.1-ubuntu-jammy                        "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               nova_ssh
07a3cf7350ea   quay.io/openstack.kolla/nova-novncproxy:2023.1-ubuntu-jammy                 "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               nova_novncproxy
305ec07b2ffe   quay.io/openstack.kolla/nova-conductor:2023.1-ubuntu-jammy                  "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               nova_conductor
939758047eff   quay.io/openstack.kolla/nova-api:2023.1-ubuntu-jammy                        "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               nova_api
6aa478ddbfdf   quay.io/openstack.kolla/nova-scheduler:2023.1-ubuntu-jammy                  "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               nova_scheduler
9bdc70ca31e5   quay.io/openstack.kolla/placement-api:2023.1-ubuntu-jammy                   "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               placement_api
40a1dc539a23   quay.io/openstack.kolla/cinder-backup:2023.1-ubuntu-jammy                   "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               cinder_backup
5f53ed873469   quay.io/openstack.kolla/cinder-volume:2023.1-ubuntu-jammy                   "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               cinder_volume
ff2dd90e8b31   quay.io/openstack.kolla/cinder-scheduler:2023.1-ubuntu-jammy                "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               cinder_scheduler
2690c886d94c   quay.io/openstack.kolla/cinder-api:2023.1-ubuntu-jammy                      "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               cinder_api
a3b81154150c   quay.io/openstack.kolla/glance-api:2023.1-ubuntu-jammy                      "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               glance_api
7f00a3ced16f   quay.io/openstack.kolla/keystone:2023.1-ubuntu-jammy                        "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               keystone
ac914ebf9cfd   quay.io/openstack.kolla/keystone-fernet:2023.1-ubuntu-jammy                 "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               keystone_fernet
f8caf2ab7aeb   quay.io/openstack.kolla/keystone-ssh:2023.1-ubuntu-jammy                    "dumb-init --single-…"   About an hour ago   Up About an hour (healthy)               keystone_ssh
31a42fa776e4   quay.io/openstack.kolla/etcd:2023.1-ubuntu-jammy                            "dumb-init --single-…"   2 hours ago         Up 2 hours                               etcd
c13e1e6d89c9   quay.io/openstack.kolla/rabbitmq:2023.1-ubuntu-jammy                        "dumb-init --single-…"   2 hours ago         Up 2 hours (healthy)                     rabbitmq
5626fb424944   quay.io/openstack.kolla/tgtd:2023.1-ubuntu-jammy                            "dumb-init --single-…"   2 hours ago         Up 2 hours                               tgtd
eeab99566682   quay.io/openstack.kolla/iscsid:2023.1-ubuntu-jammy                          "dumb-init --single-…"   2 hours ago         Up 2 hours                               iscsid
0d3b12c83a87   quay.io/openstack.kolla/prometheus-libvirt-exporter:2023.1-ubuntu-jammy     "dumb-init --single-…"   2 hours ago         Up 2 hours                               prometheus_libvirt_exporter
77817da7fccf   quay.io/openstack.kolla/prometheus-blackbox-exporter:2023.1-ubuntu-jammy    "dumb-init --single-…"   2 hours ago         Up 2 hours                               prometheus_blackbox_exporter
1178d04c31dd   quay.io/openstack.kolla/prometheus-openstack-exporter:2023.1-ubuntu-jammy   "dumb-init --single-…"   2 hours ago         Up About an hour                         prometheus_openstack_exporter
a354fb28a485   quay.io/openstack.kolla/prometheus-alertmanager:2023.1-ubuntu-jammy         "dumb-init --single-…"   2 hours ago         Up 2 hours                               prometheus_alertmanager
8085041716a9   quay.io/openstack.kolla/prometheus-cadvisor:2023.1-ubuntu-jammy             "dumb-init --single-…"   2 hours ago         Up 2 hours                               prometheus_cadvisor
f010a617f5e5   quay.io/openstack.kolla/prometheus-memcached-exporter:2023.1-ubuntu-jammy   "dumb-init --single-…"   2 hours ago         Up 2 hours                               prometheus_memcached_exporter
5832099a963c   quay.io/openstack.kolla/prometheus-mysqld-exporter:2023.1-ubuntu-jammy      "dumb-init --single-…"   2 hours ago         Up 2 hours                               prometheus_mysqld_exporter
9047fede582e   quay.io/openstack.kolla/prometheus-node-exporter:2023.1-ubuntu-jammy        "dumb-init --single-…"   2 hours ago         Up 2 hours                               prometheus_node_exporter
410178114e97   quay.io/openstack.kolla/prometheus-v2-server:2023.1-ubuntu-jammy            "dumb-init --single-…"   2 hours ago         Up 2 hours                               prometheus_server
0382e012c0aa   quay.io/openstack.kolla/memcached:2023.1-ubuntu-jammy                       "dumb-init --single-…"   2 hours ago         Up 2 hours (healthy)                     memcached
2f89e7608640   quay.io/openstack.kolla/mariadb-server:2023.1-ubuntu-jammy                  "dumb-init -- kolla_…"   2 hours ago         Up 2 hours (healthy)                     mariadb
aab537145f54   quay.io/openstack.kolla/cron:2023.1-ubuntu-jammy                            "dumb-init --single-…"   2 hours ago         Up 2 hours                               cron
1ab9fa3781e0   quay.io/openstack.kolla/kolla-toolbox:2023.1-ubuntu-jammy                   "dumb-init --single-…"   2 hours ago         Up 2 hours                               kolla_toolbox
1aa1e60be618   quay.io/openstack.kolla/fluentd:2023.1-ubuntu-jammy                         "dumb-init --single-…"   2 hours ago         Up 2 hours                               fluentd

All-in-one OpenStack is now up and running.

Install OpenStack Command Line tools

Install OpenStack command line administration tools. You can do this from the virtual environment.

source $HOME/kolla-openstack/bin/activate
pip install python-openstackclient -c https://releases.openstack.org/constraints/upper/2023.1
pip install python-neutronclient -c https://releases.openstack.org/constraints/upper/2023.1
pip install python-glanceclient -c https://releases.openstack.org/constraints/upper/2023.1
pip install python-heatclient -c https://releases.openstack.org/constraints/upper/2023.1

Generate OpenStack Admin Credentials

Generate OpenStack admin user credentials file (openrc) using the command below

kolla-ansible post-deploy

This command generates the admin credentials file, /etc/kolla/admin-openrc.sh.

To be able to use OpenStack command line tools, you need to activate the credentials using the command below;

source /etc/kolla/admin-openrc.sh

You can now administer OpenStack from cli. For example, to list the currently enabled services;

openstack service list
+----------------------------------+-----------+----------------+
| ID                               | Name      | Type           |
+----------------------------------+-----------+----------------+
| 0f263a1ae9434366b99e426c1c8abbe9 | glance    | image          |
| 33f79492fb1e43db867a1da30b871f58 | gnocchi   | metric         |
| 734aec8dcb2a4a0aa05de6a2c1a122df | heat-cfn  | cloudformation |
| 85b2d1bf703148889c0bd5a4f8092d5b | nova      | compute        |
| 88f54f6b8d95430a947eb77a2b5df010 | heat      | orchestration  |
| 8b583e0f6658454882e39364f71c61ae | aodh      | alarming       |
| a8ce69c985664a7fbaebb0207c8db850 | neutron   | network        |
| de1ac58938bc41b19ddf3cc7c60020af | cinderv3  | volumev3       |
| efeb82a116174c94a84402795b1896db | keystone  | identity       |
| f03b2a2915094d61a7458681c5d54a97 | placement | placement      |
+----------------------------------+-----------+----------------+

Initialize OpenStack [Optional]

There is an OPTIONAL script that you can execute to initialize OpenStack by creating example networks, images, nova keys using init-runonce script. The script downloads a cirros image and registers it. Then it configures networking and nova quotas to allow 40 m1.small instances to be created.

If you want to use this script, then update your networking by editing the init-runonce script and configure your public network,that you want to connect to the internet via.

vim kolla-openstack/share/kolla-ansible/init-runonce
...
ENABLE_EXT_NET=${ENABLE_EXT_NET:-1}
EXT_NET_CIDR=${EXT_NET_CIDR:-'192.168.122.0/24'}
EXT_NET_RANGE=${EXT_NET_RANGE:-'start=192.168.122.50,end=192.168.122.80'}
EXT_NET_GATEWAY=${EXT_NET_GATEWAY:-'192.168.122.1'}

Where:

  • ENABLE_EXT_NET: This variable controls whether or not the external network is enabled. If this variable is set to 0, the external network will not be enabled.
  • EXT_NET_CIDR: This variable specifies the CIDR block for the external network. The CIDR block is a way of specifying a range of IP addresses.
  • EXT_NET_RANGE: This variable specifies the range of IP addresses that are available for the external network. The range of IP addresses is specified using the start and end parameters.
  • EXT_NET_GATEWAY: This variable specifies the gateway for the external network. The gateway is the IP address of the router that connects the external network to the internet.

Next, run the script from the virtual environment.

source $HOME/kolla-openstack/bin/activate
kolla-openstack/share/kolla-ansible/init-runonce
...
+----------------------------+----------+
| Field                      | Value    |
+----------------------------+----------+
| OS-FLV-DISABLED:disabled   | False    |
| OS-FLV-EXT-DATA:ephemeral  | 0        |
| description                | None     |
| disk                       | 80       |
| id                         | 4        |
| name                       | m1.large |
| os-flavor-access:is_public | True     |
| properties                 |          |
| ram                        | 8192     |
| rxtx_factor                | 1.0      |
| swap                       |          |
| vcpus                      | 4        |
+----------------------------+----------+
+----------------------------+-----------+
| Field                      | Value     |
+----------------------------+-----------+
| OS-FLV-DISABLED:disabled   | False     |
| OS-FLV-EXT-DATA:ephemeral  | 0         |
| description                | None      |
| disk                       | 160       |
| id                         | 5         |
| name                       | m1.xlarge |
| os-flavor-access:is_public | True      |
| properties                 |           |
| ram                        | 16384     |
| rxtx_factor                | 1.0       |
| swap                       |           |
| vcpus                      | 8         |
+----------------------------+-----------+

Done.

To deploy a demo instance, run:

openstack --os-cloud=kolla-admin server create \
    --image cirros \
    --flavor m1.tiny \
    --key-name mykey \
    --network demo-net \
    demo1

Once done, you can confirm some of the things,e.g list networks created so far;

source /etc/kolla/admin-openrc.sh
openstack network list
+--------------------------------------+----------+--------------------------------------+
| ID                                   | Name     | Subnets                              |
+--------------------------------------+----------+--------------------------------------+
| 709e152e-88b5-4283-9697-43c004a8db52 | public1  | 14b9dffb-e424-4e41-ac12-6ea91a1939f9 |
| 75b0cb7e-eeec-4dce-b855-169db18dcb7d | demo-net | b27c2a65-1e52-4c55-808a-09b5fa17965e |
+--------------------------------------+----------+--------------------------------------+

Re-configuring the Stack

If you want to reconfigure the stack by adding or removing services, edit the globals.yml configuration file and re-deploy the changes from the virtual environment.

For example, after making changes on the globals.yaml config file, reconfigure the stack;

source /path/to/virtual-environment/bin/activate

The redeploy the changes;

kolla-ansible -i all-in-one reconfigure

Accessing OpenStack Web Interface (Horizon)

So far so good! OpenStack is up and running. It is time we login to the web interface.

First, check the OpenStack IP address (the Kolla VIP address, we set it before to our node ).

ip add show enp1s0
2: enp1s0:  mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:fd:a1:24 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.100/24 brd 192.168.122.255 scope global enp1s0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fefd:a124/64 scope link 
       valid_lft forever preferred_lft forever

So, 192.168.122.100, is the IP address with which we access OpenStack from the external browser.

Therefore, to access the OpenStack Horizon from the browser, use the address, http://192.168.122.100.

.This should take you to OpenStack web interface login page;

Deploy All-In-One OpenStack with Kolla-Ansible on Ubuntu 22.04

Login using admin as the username.

You can obtain the admin credentials from the Kolla passwords file, /etc/kolla/passwords.yml. For the Horizon authentication, you need to the Keystone admin password.

grep keystone_admin_password /etc/kolla/passwords.yml
keystone_admin_password: NWKXF22j9DvXq3HCtBijGcjw8pjUfWtIqWpoM7LV

When you successfully log in, you land on OpenStack horizon dashboard.

Deploy All-In-One OpenStack with Kolla-Ansible on Ubuntu 22.04

OpenStack Images

We already have cirros image registered (Admin > Compute > Images). You can also check from Project section.

openstack images

You can list images in the command line using the command below;

source $HOME/kolla-openstack/bin/activate
source /etc/kolla/admin-openrc.sh
openstack image list

Sample output;

+--------------------------------------+--------+--------+
| ID                                   | Name   | Status |
+--------------------------------------+--------+--------+
| 65fbea4f-821e-4f18-a6cc-4b46fcdcf1a6 | cirros | active |
+--------------------------------------+--------+--------+

OpenStack Image Flavors

We also have different flavors of the cirros image created;

Deploy All-In-One OpenStack with Kolla-Ansible on Ubuntu 22.04

OpenStack Networks

Example networks (Admin > Network > Networks) created. You can also check from Project section.

openstack networks

Launch OpenStack Instance

To create and launch an instance, navigate to Project > Compute > Instances. Click Launch Instance.

Set the details of the instance, set the source image, the flavor, the networks and other settings.

launch openstack instance details

Click Launch Instance when done.

The instance takes a few mins to create.

Deploy All-In-One OpenStack with Kolla-Ansible on Ubuntu 22.04

When the instance fully launches, click on its name to see more details including logs, access to console...

openstack instance details

The console;

cirros console

You can as well deploy an instance using OpenStack CLI client;

openstack server create \
    --image cirros \
    --flavor m1.tiny \
    --key-name mykey \
    --network demo-net \
    inst002

You can list key pairs using the command;

openstack keys list

Check the status of the OpenStack instances;

openstack server list
+--------------------------------------+-----------------+--------+--------------------+--------------------------+---------+
| ID                                   | Name            | Status | Networks           | Image                    | Flavor  |
+--------------------------------------+-----------------+--------+--------------------+--------------------------+---------+
| 5b8cdb51-44d8-4291-86e1-66729a9ce5ad | cirros-instance | ACTIVE | demo-net=10.0.0.37 | N/A (booted from volume) | m1.tiny |
+--------------------------------------+-----------------+--------+--------------------+--------------------------+---------+

For more OpenStack commands, refer to;

OpenStack command-line interface cheat sheet

And that marks the end of our guide on how to deploy all-in-one OpenStack with Kolla-Ansible on Ubuntu 22.04.

Further Reading

Create and Upload Custom Linux Image into OpenStack

OpenStack Administration guides

Getting Started with Docker

Reference

OpenStack Kolla-Ansible Quick Start Guide

SUPPORT US VIA A VIRTUAL CUP OF COFFEE

We're passionate about sharing our knowledge and experiences with you through our blog. If you appreciate our efforts, consider buying us a virtual coffee. Your support keeps us motivated and enables us to continually improve, ensuring that we can provide you with the best content possible. Thank you for being a coffee-fueled champion of our work!

Photo of author
Kifarunix
Linux Certified Engineer, with a passion for open-source technology and a strong understanding of Linux systems. With experience in system administration, troubleshooting, and automation, I am skilled in maintaining and optimizing Linux infrastructure.

Related Posts

Install and Setup OpenLDAP Server on Debian 12

Install and Setup OpenLDAP Server on Debian 11

How to Install LAMP Stack (Apache,MariaDB, PHP 7.2) on Ubuntu 18.04 LTS

How to Install Programs from Source on Ubuntu 18.04

The Rise of Digital Technology in Education: How to Benefit From it

Install Apache Guacamole on Rocky Linux 8

Leave a Comment

© 2023 kifarunix.com

Home Advertise with us Privacy Policy