The WolfspyreLabs Blog/ 2023/ Posts from July/ More fun with Ceph RADOSGW/ Updating Privileges/ Updating Privileges for CLABEL in radosgw; do TARGET="/etc/pve/priv/ceph.client.${CLABEL}.keyring"; for H in 40 41 42 43 44 45; do CLIENT=“client.radosgw.px-m-${H}” ceph-authtool -n ${CLIENT} –cap osd ‘allow rwx’ –cap mon ‘allow rwx’ –cap mon ‘allow command “config-key get” with “key” prefix “rgw”’ ${TARGET}; echo “added privileges to ${CLIENT} in ${TARGET}”; done; done ceph auth caps client.radosgw.px-m-40 osd ‘allow rwx’ mon ‘allow rwx, allow command “config-key get” with “key” prefix “rgw”, allow command “config get” with “key” prefix “client”’ ceph auth caps <iscsi_daemon_client> mon ‘profile rbd, allow command “osd blocklist”, allow command “config-key get” with “key” prefix “iscsi/”’ mgr ‘allow *’ osd ‘allow rwx’ ceph auth caps client.radosgw.px-m-40 mon ‘allow command “config-key get” with “key” prefix “rgw”’ ==> ceph.audit.log <== 2023-07-09T21:12:11.500895-0500 mon.px-m-44 (mon.4) 15754 : audit [DBG] from=‘mgr.287705578 198.18.198.45:0/1423427’ entity=‘mgr.px-m-45’ cmd=[{“prefix”: “osd blocklist ls”, “format”: “json”}]: dispatch 2023-07-09T21:12:11.818644-0500 mon.px-m-41 (mon.2) 8498 : audit [INF] from=‘client.? 198.18.198.41:0/2228261558’ entity=‘client.admin’ cmd=[{“prefix”: “auth caps”, “entity”: “client.radosgw.px-m-40”, “caps”: [“mon”, “allow command "config-key get" with "key" prefix "rgw"”]}]: dispatch 2023-07-09T21:12:11.819641-0500 mon.px-m-43 (mon.0) 6854 : audit [INF] from=‘client.? ’ entity=‘client.admin’ cmd=[{“prefix”: “auth caps”, “entity”: “client.radosgw.px-m-40”, “caps”: [“mon”, “allow command "config-key get" with "key" prefix "rgw"”]}]: dispatch 2023-07-09T21:12:11.858173-0500 mon.px-m-43 (mon.0) 6855 : audit [INF] from=‘client.? ’ entity=‘client.admin’ cmd=’[{“prefix”: “auth caps”, “entity”: “client.radosgw.px-m-40”, “caps”: [“mon”, “allow command "config-key get" with "key" prefix "rgw"”]}]’: finished 2023-07-09T21:12:13.183663-0500 mon.px-m-40 (mon.3) 7411 : audit [DBG] from=‘client.? 198.18.198.42:0/1076355190’ entity=‘client.admin’ cmd=[{“prefix”:“df”,“format”:“json”}]: dispatch Jul 09 22:34:33 px-m-40 radosgw[2896582]: ignoring –setuser ceph since I am not root Jul 09 22:34:33 px-m-40 radosgw[2896582]: ignoring –setgroup ceph since I am not root Jul 09 22:34:33 px-m-40 radosgw[2896582]: 2023-07-09T22:34:33.491-0500 7f99e71bda00 -1 Errors while parsing config file! Jul 09 22:34:33 px-m-40 radosgw[2896582]: 2023-07-09T22:34:33.491-0500 7f99e71bda00 -1 can’t open ceph.conf: (2) No such file or directory Jul 09 22:34:33 px-m-40 radosgw[2896582]: unable to get monitor info from DNS SRV with service name: ceph-mon Jul 09 22:34:33 px-m-40 radosgw[2896582]: 2023-07-09T22:34:33.495-0500 7f99e71bda00 -1 failed for service _ceph-mon._tcp Jul 09 22:34:33 px-m-40 radosgw[2896582]: 2023-07-09T22:34:33.495-0500 7f99e71bda00 -1 monclient: get_monmap_and_config cannot identify monitors to contact ceph auth get client.radosgw.px-m-40 [client.radosgw.px-m-40] key = xxxxxxx caps mon = "allow rwx, allow command \"config-key get\" with \"key\" prefix \"rgw\", allow command \"config get\" " caps osd = "allow rwx" alternatively … #ceph auth caps client.radosgw.px-m-41 osd 'allow rwx' mon "allow rwx, allow command 'config-key get' with 'key' prefix 'rgw', allow command 'config get'" which looks like: [client.radosgw.px-m-41] key = AQDESdNjXH5JEhAAnB3vWs7zLbQdK9Z2jC9I0g== caps mon = "allow rwx, allow command 'config-key get' with 'key' prefix 'rgw', allow command 'config get'" caps osd = "allow rwx" and works as expected: root@px-m-41:/var/log/ceph# ceph --user radosgw.px-m-41 --keyring /etc/ceph/ceph.client.radosgw.keyring config get client rgw_frontends beast port=7480 ssl_port=7443 ssl_certificate=config://rgw/cert/noblewise/default.crt ssl_private_key=config://rgw/cert/noblewise/default.key root@px-m-44:/var/log/ceph# for _H in 40 41 42 43 44 45; do ceph auth caps client.radosgw.px-m-${_H} osd ‘allow rwx’ mon “allow rwx, allow command ‘config-key get’ with ‘key’ prefix ‘rgw’, allow command ‘config get’";done updated caps for client.radosgw.px-m-40 updated caps for client.radosgw.px-m-41 updated caps for client.radosgw.px-m-42 updated caps for client.radosgw.px-m-43 updated caps for client.radosgw.px-m-44 ==> ceph.audit.log <== 2023-07-09T20:40:22.419040-0500 mon.px-m-45 (mon.5) 20601 : audit [DBG] from='client.? 198.18.198.40:0/380683984' entity='client.radosgw.px-m-40' cmd=[{"prefix": "config-key get", "key": "rgw/cert/noblewise/default.key"}]: access denied 2023-07-09T20:40:22.420044-0500 mon.px-m-45 (mon.5) 20602 : audit [DBG] from='client.? 198.18.198.40:0/380683984' entity='client.radosgw.px-m-40' cmd=[{"prefix": "config-key get", "key": "rgw/cert/noblewise/default.crt"}]: access denied https://tracker.ceph.com/issues/48925 https://docs.ceph.com/en/quincy/radosgw/troubleshooting/