Tkrzw
|
File database manager implementation based on hash table. More...
#include <tkrzw_dbm_hash.h>
Classes | |
class | Iterator |
Iterator for each record. More... | |
struct | TuningParameters |
Tuning parameters for the database. More... | |
Public Types | |
enum | UpdateMode { UPDATE_DEFAULT = 0, UPDATE_IN_PLACE = 1, UPDATE_APPENDING = 2 } |
Enumeration for update modes. More... | |
![]() | |
typedef std::function< std::string_view(std::string_view, std::string_view)> | RecordLambdaType |
Lambda function type to process a record. More... | |
Public Member Functions | |
HashDBM () | |
Default constructor. More... | |
HashDBM (std::unique_ptr< File > file) | |
Constructor with a file object. More... | |
virtual | ~HashDBM () |
Destructor. More... | |
HashDBM (const HashDBM &rhs)=delete | |
Copy and assignment are disabled. More... | |
HashDBM & | operator= (const HashDBM &rhs)=delete |
Status | Open (const std::string &path, bool writable, int32_t options=File::OPEN_DEFAULT) override |
Opens a database file. More... | |
Status | OpenAdvanced (const std::string &path, bool writable, int32_t options=File::OPEN_DEFAULT, const TuningParameters &tuning_params=TuningParameters()) |
Opens a database file, in an advanced way. More... | |
Status | Close () override |
Closes the database file. More... | |
Status | Process (std::string_view key, RecordProcessor *proc, bool writable) override |
Processes a record with a processor. More... | |
Status | ProcessEach (RecordProcessor *proc, bool writable) override |
Processes each and every record in the database with a processor. More... | |
Status | Count (int64_t *count) override |
Gets the number of records. More... | |
Status | GetFileSize (int64_t *size) override |
Gets the current file size of the database. More... | |
Status | GetFilePath (std::string *path) override |
Gets the path of the database file. More... | |
Status | Clear () override |
Removes all records. More... | |
Status | Rebuild () override |
Rebuilds the entire database. More... | |
Status | RebuildAdvanced (const TuningParameters &tuning_params=TuningParameters(), bool skip_broken_records=false) |
Rebuilds the entire database, in an advanced way. More... | |
Status | ShouldBeRebuilt (bool *tobe) override |
Checks whether the database should be rebuilt. More... | |
Status | Synchronize (bool hard, FileProcessor *proc=nullptr) override |
Synchronizes the content of the database to the file system. More... | |
std::vector< std::pair< std::string, std::string > > | Inspect () override |
Inspects the database. More... | |
bool | IsOpen () const override |
Checks whether the database is open. More... | |
bool | IsWritable () const override |
Checks whether the database is writable. More... | |
bool | IsHealthy () const override |
Checks whether the database condition is healthy. More... | |
bool | IsOrdered () const override |
Checks whether ordered operations are supported. More... | |
std::unique_ptr< DBM::Iterator > | MakeIterator () override |
Makes an iterator for each record. More... | |
std::unique_ptr< DBM > | MakeDBM () const override |
Makes a new DBM object of the same concrete class. More... | |
const File * | GetInternalFile () const |
Gets the pointer to the internal file object. More... | |
int64_t | GetEffectiveDataSize () |
Gets the effective data size. More... | |
double | GetModificationTime () |
Gets the last modification time of the database. More... | |
int32_t | GetDatabaseType () |
Gets the database type. More... | |
Status | SetDatabaseType (uint32_t db_type) |
Sets the database type. More... | |
std::string | GetOpaqueMetadata () |
Gets the opaque metadata. More... | |
Status | SetOpaqueMetadata (const std::string &opaque) |
Sets the opaque metadata. More... | |
int64_t | CountBuckets () |
Gets the number of buckets of the hash table. More... | |
int64_t | CountUsedBuckets () |
Gets the number of used buckets of the hash table. More... | |
UpdateMode | GetUpdateMode () |
Gets the current update mode. More... | |
Status | SetUpdateModeAppending () |
Set the update mode appending. More... | |
Status | ImportFromFileForward (const std::string &path, bool skip_broken_records, int64_t record_base, int64_t end_offset) |
Imports records from another hash database file, in a forward manner. More... | |
Status | ImportFromFileBackward (const std::string &path, bool skip_broken_records, int64_t record_base, int64_t end_offset) |
Imports records from another hash database file, in a backward manner. More... | |
![]() | |
virtual | ~DBM ()=default |
Destructor. More... | |
virtual Status | Process (std::string_view key, RecordLambdaType rec_lambda, bool writable) |
Processes a record with a lambda function. More... | |
virtual Status | Get (std::string_view key, std::string *value=nullptr) |
Gets the value of a record of a key. More... | |
virtual std::string | GetSimple (std::string_view key, std::string_view default_value="") |
Gets the value of a record of a key, in a simple way. More... | |
virtual std::map< std::string, std::string > | GetMulti (const std::initializer_list< std::string > &keys) |
Gets the values of multiple records of keys. More... | |
virtual std::map< std::string, std::string > | GetMulti (const std::vector< std::string > &keys) |
Gets the values of multiple records of keys, with a vector. More... | |
virtual Status | Set (std::string_view key, std::string_view value, bool overwrite=true, std::string *old_value=nullptr) |
Sets a record of a key and a value. More... | |
virtual Status | SetMulti (const std::initializer_list< std::pair< std::string, std::string >> &records, bool overwrite=true) |
Sets multiple records. More... | |
virtual Status | SetMulti (const std::map< std::string, std::string > &records, bool overwrite=true) |
Sets multiple records, with a map of strings. More... | |
virtual Status | Remove (std::string_view key, std::string *old_value=nullptr) |
Removes a record of a key. More... | |
virtual Status | Append (std::string_view key, std::string_view value, std::string_view delim="") |
Appends data at the end of a record of a key. More... | |
virtual Status | CompareExchange (std::string_view key, std::string_view expected, std::string_view desired, std::string *actual=nullptr) |
Compares the value of a record and exchanges if the condition meets. More... | |
virtual Status | Increment (std::string_view key, int64_t increment=1, int64_t *current=nullptr, int64_t initial=0) |
Increments the numeric value of a record. More... | |
int64_t | IncrementSimple (std::string_view key, int64_t increment=1, int64_t initial=0) |
Increments the numeric value of a record, in a simple way. More... | |
virtual Status | ProcessEach (RecordLambdaType rec_lambda, bool writable) |
Processes each and every record in the database with a lambda function. More... | |
virtual int64_t | CountSimple () |
Gets the number of records, in a simple way. More... | |
virtual int64_t | GetFileSizeSimple () |
Gets the current file size of the database, in a simple way. More... | |
virtual std::string | GetFilePathSimple () |
Gets the path of the database file, in a simple way. More... | |
virtual bool | ShouldBeRebuiltSimple () |
Checks whether the database should be rebuilt, in a simple way. More... | |
virtual Status | CopyFileData (const std::string &dest_path) |
Copies the content of the database file to another file. More... | |
virtual Status | Export (DBM *dbm) |
Exports all records to another database. More... | |
Static Public Member Functions | |
static Status | FindRecordBase (File *file, int64_t *record_base, int32_t *offset_width, int32_t *align_pow, int64_t *last_sync_size) |
Finds the record base of a hash database file. More... | |
static Status | RestoreDatabase (const std::string &old_file_path, const std::string &new_file_path, int64_t end_offset) |
Restores a broken database as a new healthy database. More... | |
Static Public Attributes | |
static constexpr int32_t | DEFAULT_OFFSET_WIDTH = 4 |
The default value of the offset width. More... | |
static constexpr int32_t | DEFAULT_ALIGN_POW = 3 |
The default value of the alignment power. More... | |
static constexpr int64_t | DEFAULT_NUM_BUCKETS = 1048583 |
The default value of the number of buckets. More... | |
static constexpr int32_t | DEFAULT_FBP_CAPACITY = 2048 |
The default value of the capacity of the free block pool. More... | |
static constexpr int32_t | OPAQUE_METADATA_SIZE = 64 |
The size of the opaque metadata. More... | |
File database manager implementation based on hash table.
All operations are thread-safe; Multiple threads can access the same database concurrently. Every opened database must be closed explicitly to avoid data corruption.
tkrzw::HashDBM::HashDBM | ( | ) |
Default constructor.
MemoryMapParallelFile is used to handle the data.
tkrzw::HashDBM::HashDBM | ( | std::unique_ptr< File > | file | ) |
Constructor with a file object.
file | The file object to handle the data. The ownership is taken. |
|
virtual |
Destructor.
|
explicitdelete |
Copy and assignment are disabled.
|
overridevirtual |
Opens a database file.
path | A path of the file. |
writable | If true, the file is writable. If false, it is read-only. |
options | Bit-sum options of File::OpenOption enums for opening the file. |
Implements tkrzw::DBM.
Status tkrzw::HashDBM::OpenAdvanced | ( | const std::string & | path, |
bool | writable, | ||
int32_t | options = File::OPEN_DEFAULT , |
||
const TuningParameters & | tuning_params = TuningParameters() |
||
) |
Opens a database file, in an advanced way.
path | A path of the file. |
writable | If true, the file is writable. If false, it is read-only. |
options | Bit-sum options for opening the file. |
tuning_params | A structure for tuning parameters. |
|
overridevirtual |
Closes the database file.
Implements tkrzw::DBM.
|
overridevirtual |
Processes a record with a processor.
key | The key of the record. |
proc | The pointer to the processor object. |
writable | True if the processor can edit the record. |
Implements tkrzw::DBM.
|
overridevirtual |
Processes each and every record in the database with a processor.
proc | The pointer to the processor object. |
writable | True if the processor can edit the record. |
Implements tkrzw::DBM.
|
overridevirtual |
Gets the number of records.
count | The pointer to an integer object to contain the result count. |
Implements tkrzw::DBM.
|
overridevirtual |
Gets the current file size of the database.
size | The pointer to an integer object to contain the result size. |
Implements tkrzw::DBM.
|
overridevirtual |
Gets the path of the database file.
path | The pointer to a string object to contain the result path. |
Implements tkrzw::DBM.
|
overridevirtual |
Removes all records.
Implements tkrzw::DBM.
|
overridevirtual |
Rebuilds the entire database.
Implements tkrzw::DBM.
Status tkrzw::HashDBM::RebuildAdvanced | ( | const TuningParameters & | tuning_params = TuningParameters() , |
bool | skip_broken_records = false |
||
) |
Rebuilds the entire database, in an advanced way.
tuning_params | A structure for tuning parameters. The default value of each parameter means that the current setting is succeeded or calculated implicitly. |
skip_broken_records | If true, the operation continues even if there are broken records which can be skipped. |
|
overridevirtual |
Checks whether the database should be rebuilt.
tobe | The pointer to a boolean object to contain the result decision. |
Implements tkrzw::DBM.
|
overridevirtual |
Synchronizes the content of the database to the file system.
hard | True to do physical synchronization with the hardware or false to do only logical synchronization with the file system. |
proc | The pointer to the file processor object, whose Process method is called while the content of the file is synchronized. If it is nullptr, it is ignored. |
Implements tkrzw::DBM.
|
overridevirtual |
Inspects the database.
Implements tkrzw::DBM.
|
overridevirtual |
Checks whether the database is open.
Implements tkrzw::DBM.
|
overridevirtual |
Checks whether the database is writable.
Implements tkrzw::DBM.
|
overridevirtual |
Checks whether the database condition is healthy.
Implements tkrzw::DBM.
|
overridevirtual |
Checks whether ordered operations are supported.
Implements tkrzw::DBM.
|
overridevirtual |
Makes an iterator for each record.
Implements tkrzw::DBM.
|
overridevirtual |
Makes a new DBM object of the same concrete class.
Implements tkrzw::DBM.
const File* tkrzw::HashDBM::GetInternalFile | ( | ) | const |
Gets the pointer to the internal file object.
int64_t tkrzw::HashDBM::GetEffectiveDataSize | ( | ) |
Gets the effective data size.
double tkrzw::HashDBM::GetModificationTime | ( | ) |
Gets the last modification time of the database.
int32_t tkrzw::HashDBM::GetDatabaseType | ( | ) |
Gets the database type.
Status tkrzw::HashDBM::SetDatabaseType | ( | uint32_t | db_type | ) |
Sets the database type.
db_type | The database type. |
std::string tkrzw::HashDBM::GetOpaqueMetadata | ( | ) |
Gets the opaque metadata.
Status tkrzw::HashDBM::SetOpaqueMetadata | ( | const std::string & | opaque | ) |
Sets the opaque metadata.
opaque | The opaque metadata, of which leading 64 bytes are stored in the file. |
int64_t tkrzw::HashDBM::CountBuckets | ( | ) |
Gets the number of buckets of the hash table.
int64_t tkrzw::HashDBM::CountUsedBuckets | ( | ) |
Gets the number of used buckets of the hash table.
UpdateMode tkrzw::HashDBM::GetUpdateMode | ( | ) |
Gets the current update mode.
Status tkrzw::HashDBM::SetUpdateModeAppending | ( | ) |
Set the update mode appending.
Precondition: The database is opened as writable. Whereas it is allowed to change a database in the in-place mode to the appending mode, the opposite direction is not allowed.
Status tkrzw::HashDBM::ImportFromFileForward | ( | const std::string & | path, |
bool | skip_broken_records, | ||
int64_t | record_base, | ||
int64_t | end_offset | ||
) |
Imports records from another hash database file, in a forward manner.
path | A path of the other hash database file. |
skip_broken_records | If true, the operation continues even if there are broken records which can be skipped. |
record_base | The beginning offset of records to read. Negative means the beginning of the record section. |
end_offset | The exclusive end offset of records to read. Negative means unlimited. 0 means the size when the database is synched or closed properly. |
Status tkrzw::HashDBM::ImportFromFileBackward | ( | const std::string & | path, |
bool | skip_broken_records, | ||
int64_t | record_base, | ||
int64_t | end_offset | ||
) |
Imports records from another hash database file, in a backward manner.
path | A path of the other hash database file. |
skip_broken_records | If true, the operation continues even if there are broken records which can be skipped. |
record_base | The beginning offset of records to read. Negative means the beginning of the record section. |
end_offset | The exclusive end offset of records to read. Negative means unlimited. 0 means the size when the database is synched or closed properly. |
|
static |
Finds the record base of a hash database file.
file | A file object having opened the database file. |
record_base | The pointer to an integer to store the record base offset. |
offset_width | The pointer to an integer to store the offset width. |
align_pow | The pointer to an integer to store the alignment power. |
last_sync_size | The pointer to an integer to store the file size when the database was synched or closed properly at the last time. This size is zero, if the metadata is broken. |
|
static |
Restores a broken database as a new healthy database.
old_file_path | The path of the broken database. |
new_file_path | The path of the new database to be created. |
end_offset | The exclusive end offset of records to read. Negative means unlimited. 0 means the size when the database is synched or closed properly. |
|
staticconstexpr |
The default value of the offset width.
|
staticconstexpr |
The default value of the alignment power.
|
staticconstexpr |
The default value of the number of buckets.
|
staticconstexpr |
The default value of the capacity of the free block pool.
|
staticconstexpr |
The size of the opaque metadata.