The RedHat Production Grade OSG Guide1 has a LOT of useful info. I’d strongly encourage you to take some time to look through it to familiarize yourself with the moving pieces.
There’s a LOT going on in there.
TARGET='/etc/pve/priv/ceph.client.radosgw.keyring'for H in 4041424344 45;doCLIENT="client.radosgw.px-m-${H}" ceph-authtool -n ${CLIENT} --cap osd 'allow rwx' --cap mon 'allow rwx'${TARGET}echo"added privileges to ${CLIENT} in ${TARGET}"done
Add the newly minted auth tokens to the cluster
Using the admin keyring, add the newly minted tokens to the cluster.
Add the new keys to the cluster
ADMINKEY='/etc/pve/priv/ceph.client.admin.keyring'TARGET='/etc/pve/priv/ceph.client.radosgw.keyring'for H in 4041424344 45;doCLIENT="client.radosgw.px-m-${H}" ceph -k ${ADMINKEY} auth add ${CLIENT} -i ${TARGET}done
Output
added key for client.radosgw.px-m-40
added key for client.radosgw.px-m-41
added key for client.radosgw.px-m-42
added key for client.radosgw.px-m-43
added key for client.radosgw.px-m-44
added key for client.radosgw.px-m-45
Config
Update /etc/services
Adding RADOSgw to /etc/services makes various systemic tools aware of the most likely use is, and displays the service in question when viewing nework connection states./etc/services
radosgw 7480/tcp # Ceph Rados gw
Adjusting Thread Cache Memory
The RH Guidance2 is to adjust Ceph’s TCMalloc setting to tune how much memory is allocated for ceph’s thread cache..
in RHEL/CentOS this is adjusted in /etc/sysconfig/ceph. However, ProxMox is based on Debian / Ubuntu The “default” config dir there is /etc/default/ as such the file to inspect is /etc/default/ceph.
When I looked, it was already set to what I believe to be an acceptable level:
Inspecting Ceph TCMalloc setting
root@px-m-41:/tmp/ceph-px-m-41# more /etc/default/ceph
# /etc/default/ceph
#
# Environment file for ceph daemon systemd unit files.
#
# Increase tcmalloc cache size
TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728
Increase Systemic Limits
The RH Guidance3 is to increase the file descriptor limits for the ceph user in /etc/security/limits.conf:
/etc/security/limits.conf
ceph soft nproc unlimited
Adjust ceph config file
Insert the following blob into the [global] section of /etc/ceph/ceph.conf:add to the global section of /etc/ceph/ceph.conf
I thought about setting these in the config file, but left them at their defaults (most of which undefined)add to the global section of /etc/ceph/ceph.conf