Skip to main content

Bucket Metrics

The Metric Exporter feature is designed to simplify collecting and analyzing bucket data and helps you quickly and efficiently gain useful insight into your bucket's performance, behavior, and errors. This tool automatically processes your logs and generates various metrics that you can use to build charts and reports in tools such as Grafana, Prometheus, and Splunk.

The data is sent in Plain Text format.

Bucket metrics work differently from log forwarding. Because of their large volume, logs need more storage space, while metrics are structured and lighter. In addition, logs are mostly used for troubleshooting and analyzing event details, while metrics are suitable for monitoring, alerting, and analyzing patterns and trends.

Exported Metrics​

With bucket metrics, you can receive your buckets' Access Metrics.

Access MetricDescriptionTypeLabelsUpdate Interval
storage_bucket_operation_duration_msFrequency distribution of bucket operation duration (Get, Put, etc.) in millisecondshistograminstance, bucket, operation, le60 seconds
storage_bucket_object_countTotal number of objects currently stored in the bucketgaugeinstance, bucket60 seconds
storage_bucket_size_bytesTotal size of objects stored in the bucket, in bytesgaugeinstance, bucket60 seconds
storage_bucket_send_bytes_totalCumulative total of bytes sent from the bucket (such as downloads and reads)counterinstance, bucket60 seconds
storage_bucket_receive_bytes_totalCumulative total of bytes received by the bucket (such as uploads and writes)counterinstance, bucket60 seconds

Managing Bucket Metrics in the Panel​

In the Arvancloud panel, go to Object Storage and select Bucket Metrics.

Click + Metric.

On this page, select the bucket whose metrics you want to receive, the log type, and the collection interval, then click Save.

You can then enable or disable metrics, edit and manage them, or delete them.

Managing Bucket Metrics via API​

To work with the bucket metrics API, see the API documentation for this feature. Before using the API, you need to create an access key in the panel. You can do this by creating and getting an access key.

Store the access key you receive in this step in a safe place.

Getting the List of Bucket Metrics​

To get the list of bucket metrics via API, you can use the following sample request:

curl --request GET \
--url https://storage.arvanapis.ir/v1/buckets/{bucketName}/metrics \
--header 'Authorization: [apikey]'


Note that you need to replace [apikey] with your API key in the format Apikey xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.

The response to this request looks like this:

# HELP storage_bucket_download_traffic_bytes Cumulative total number of bytes sent from the bucket (e.g., downloads, reads)
# TYPE storage_bucket_download_traffic_bytes counter
storage_bucket_download_traffic_bytes{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",region="ir-thr-at1"} 3.403539979e+09
# HELP storage_bucket_objects_total Current total number of objects stored in the bucket
# TYPE storage_bucket_objects_total gauge
storage_bucket_objects_total{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",region="ir-thr-at1"} 973
# HELP storage_bucket_operation_duration_milliseconds Duration in milliseconds of bucket operations (get, put, delete, list, etc.) tracked as a histogram
# TYPE storage_bucket_operation_duration_milliseconds histogram
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="5"} 0
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="10"} 0
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="25"} 0
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="50"} 1
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="100"} 2
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="250"} 3
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="500"} 3
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="1000"} 3
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="2500"} 3
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="5000"} 3
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="10000"} 3
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="30000"} 3
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="abort_multipart",region="ir-thr-at1",le="60000"} 3
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="put_obj",region="ir-thr-at1",le="60000"} 2307
storage_bucket_operation_duration_milliseconds_bucket{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="put_obj",region="ir-thr-at1",le="+Inf"} 2308
storage_bucket_operation_duration_milliseconds_sum{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="put_obj",region="ir-thr-at1"} 6.574638e+06
storage_bucket_operation_duration_milliseconds_count{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",operation="put_obj",region="ir-thr-at1"} 2308
# HELP storage_bucket_size_bytes Current total size in bytes of all objects stored in the bucket
# TYPE storage_bucket_size_bytes gauge
storage_bucket_size_bytes{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",region="ir-thr-at1"} 3.985408e+06
# HELP storage_bucket_upload_traffic_bytes Cumulative total number of bytes received by the bucket (e.g., uploads, writes)
# TYPE storage_bucket_upload_traffic_bytes counter
storage_bucket_upload_traffic_bytes{bucket="bucketname",instance="d6d2xxxx-xxxx-xxxx-xxxx-bcexxxx",region="ir-thr-at1"} 6.361366432e+09

Adding Metrics to Prometheus​

To add metrics to Prometheus, simply add this section to the prometheus.yml configuration file:

- job_name: 'Arvan_s3_metrics'
scrape_interval: 60s
scheme: https
metrics_path: '/v1/buckets/{bucketName}/metrics'
authorization:
type: Apikey
credentials: 'db94xxxx-xxxx-xxxx-xxxx-xxxx'
static_configs:
- targets:
- 'storage.arvanapis.ir'