CloudConnect Statistics Documentation

1 Scope

This document covers definitions of the various statistics produced by CloudConnect.

2 Overview

Using the elcc-cli command you will commonly see various statistics display. For example, see the status of an operation with this command

elcc-cli operation show --=1

and it will produce an output with many of the statistics in this article.


In general there are two major statistics group:

  • bucket statistics - operations done on the object store bucket (but also to the work area in the case of scan - see below)

  • file level statistics - logical files operations.


The following ELCC operations are generating meaningful stats:

  • Scan : during the scan the source file system is scanned and a Merkle tree is build in the local work area/ work space. For that purpose the work area is the bucket.

  • Commit : during the commit a new CCFS version is build in the destination bucket.

  • Checkout

  • Delete


Other commands also generate statistics, but as they are accessing the CCFS bucket in much more limited way (e.g. list dir, get attr) it is much less relevant. Such ops should be considered partial checkout.

3 File level Statistics


Stat name

Description

fs_created_files

In scan operation: Total number of files scanned in source file system 

In checkin operation: Total number of files uploaded into the bucket (and not dedup) 

In checkout operation: Total number of files updated in the target file system

fs_created_dirs

In scan operation: Total number of directories scanned in source file system 

In checkin operation: Total number of directories uploaded into the bucket (and not dedup) 

In checkout operation: Total number of directories updated in the target file system

fs_files_bytes_written

In scan operation: Total number of files data bytes scanned in source file system 

In checkin operation: Total number of non-deduped  files data bytes that were processed (either uploaded or deduped) 

In checkout operation: Total number of files data bytes updated in the target file system

fs_files_dedup

Number of files that exist in previous version in the object store and are shared by the new uploaded version- The dedup files are not uploaded to the object store (the Cloud Connect increments a refCount in the object store)

fs_dirs_dedup

Sub namespace connected to previous version. In other words the entire sub namespace is now shared between the previous version and the new one.

fs_dirs_buckets_dedup

Number of directory buckets that were dedup in commit operation - Every directory is build out of a collection of buckets. Every bucket contains a group of child file and directories. CloudConnect can dedup bucket (a group of files/dirs) in one operations (vs. dedup a file/dir one by one).

fs_files_deleted

Number of files deleted in deleted version

fs_dirs_deleted

Number of directories deleted in deleted version (and not connected to previous version)

fs_files_bytes_deleted

Number of file data bytes deleted in deleted version


4 Store Bucket level Statistics


Stat name

Description

read_blobs_bytes

Number of bytes read from store during checkin/checkout operations . This is the total bytes read by OBJECT-GET operation. This parameter is updated during checkin/checkout operations.

write_blobs_ops

number of write blob operations OBJECT-PUT This parameter is updated during checkin operation.

create_ref_count_obj_ops

Number of create reference count object operations (0 bytes size object).

write_blobs_bytes

Number of bytes written to data objects OBJECT-PUT.-This parameter is updated during checkin operation.

delete_ref_cnt_ops

Number of ref count objects  (0 bytes size object)

 Deleted -This parameter is updated during delete version operation.

delete_blobs_bytes

Capacity deleted objects by removing objects -This parameter is updated during delete version operation.

read_metadata_ops

Number of read object MD operations OBJECT-HEAD of data objects -This parameter is updated during checkin/checkout/delete-version operations.

read_ref_count_obj_ops

Number of read object MD operations OBJECT-HEAD of  ref count objects  (0 bytes size object) -This parameter is updated during checkin/checkout/delete-version operation.

read_blobs_ops

Number of read blob operations OBJECT-GET

delete_obj_ops

Number of delete blob operations OBJECT-DELETE

inc_ref_count_obj_ops

Number of ref count write blob operations OBJECT-PUT used for dedup file , a directory, a directory bucket or a data blob  - indicates total number of all dedup types




5 Requirements

5.1 Operation statistics

5.1.1 Scan

Statistics

Description

Used in GUI

Start time


Yes

Elapsed time


Yes

End time


Yes

Number of scanned dirs

Fs_created_dirs - take the number when scan ends

Add to operation “More Info” under the Scan section

Number of scanned dirs per sec

Fs_created_dirs/duration

Add to operation “More Info” under the Scan section

Number of scanned files

Fs_created_files - take the number when scan ends


Add to operation “More Info” under the Scan section

Number of scanned files per second

Fs_created_files/duration

Add to operation “More Info” under the Scan section

Scan size

fs_files_bytes_written - take the number when scan ends

Add to operation “More Info” under the Scan section

5.1.2 Commit (Check In)

Statistics

Description

Used in GUI

Start time


Yes

Elapsed time


Yes

End time


Yes

Number of bytes

fs_files_bytes_written - take the number when commit ends

Add to operation “More Info” under the Check In section

Number of bytes per second

fs_files_bytes_written/duration

Add to operation “More Info” under the Check In section

Number of files commited

Fs_created_files - take the number when commit ends


Add to operation “More Info” under the Check In section

Number of files committed per second

Fs_created_files/duration


Add to operation “More Info” under the Check In section

Number of dirs committed 

Fs_created_dirs - take the number when commit ends

Add to operation “More Info” under the Check In section

Number of dirs committed per second

Fs_created_dirs/duration

Add to operation “More Info” under the Check In section

Number of dedup files commited

fs_files_dedup

Add to operation “More Info” under the Check In section

Number of dedup files committed per second



Number of dedup dirs committed 

fs_dirs_dedup

Add to operation “More Info” under the Check In section

Number of dedup dirs committed per second



 5.1.2 Checkout

Statistics

Description

Used in GUI

Start time


Yes

Elapsed time


Yes

End time


Yes

Number of bytes

fs_files_bytes_written - take the number when check-out ends

Add to operation “More Info” under the Check-Out section

Number of bytes per second

fs_files_bytes_written/duration

Add to operation “More Info” under the Check-Out section

Number of files 

Fs_created_files - take the number when check-out ends

Add to operation “More Info” under the Check-Out section

Number of files per second

Fs_created_files/duration


Add to operation “More Info” under the Check-Out section

Number of dirs 

Fs_created_dirs - take the number when check-out ends

Add to operation “More Info” under the Check-Out section

Number of dirs per second

Fs_created_dirs/duration

Add to operation “More Info” under the Check-Out section


D
Dave is the author of this solution article.

Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.