1/* Part of SWI-Prolog 2 3 Author: Jan Wielemaker 4 E-mail: J.Wielemaker@vu.nl 5 WWW: http://www.swi-prolog.org 6 Copyright (c) 2015, University of Amsterdam 7 VU University Amsterdam 8 All rights reserved. 9 10 Redistribution and use in source and binary forms, with or without 11 modification, are permitted provided that the following conditions 12 are met: 13 14 1. Redistributions of source code must retain the above copyright 15 notice, this list of conditions and the following disclaimer. 16 17 2. Redistributions in binary form must reproduce the above copyright 18 notice, this list of conditions and the following disclaimer in 19 the documentation and/or other materials provided with the 20 distribution. 21 22 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 25 FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 26 COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 27 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 28 BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 29 LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 30 CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 31 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN 32 ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 33 POSSIBILITY OF SUCH DAMAGE. 34*/ 35 36:- module(bdb, 37 [ bdb_init/1, % +Options 38 bdb_init/2, % -Environment, +Options 39 bdb_close_environment/1, % +Environment 40 bdb_current_environment/1, % -Environment 41 bdb_environment_property/2, % ?Environment, ?Property 42 43 bdb_open/4, % +File, +Mode, -Handle, +Options 44 bdb_close/1, % +Handle 45 bdb_closeall/0, % 46 bdb_current/1, % -DB 47 48 bdb_put/3, % +DB, +Key, +Value 49 bdb_del/3, % +DB, +Key, ?Value 50 bdb_delall/3, % +DB, +Key, +Value 51 bdb_enum/3, % +DB, -Key, -Value 52 bdb_get/3, % +DB, +Key, -Value 53 bdb_getall/3, % +DB, +Key, -ValueList 54 55 bdb_transaction/1, % :Goal 56 bdb_transaction/2, % :Goal, +Environment 57 58 bdb_version/1 % -Version 59 ]). 60:- use_foreign_library(foreign(bdb4pl)). 61:- meta_predicate 62 bdb_transaction(), 63 bdb_transaction(, ). 64 65/** <module> Berkeley DB interface 66 67This package realises a binding to _Berkeley DB_, originally by 68[Sleepycat Software](http://www.sleepycat.com/), now managed by 69[Oracle](http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html). 70The DB library implements modular support for the bottom layers of a 71database. In can be configured for single-threaded access to a file, 72multi-threaded access with transactions, remote access as well as 73database replication. 74 75Berkeley DB is an _embedded_ database. This implies the library provides 76access to a file containing one or more database tables. The Berkeley DB 77database tables are always _binary_, mapping a _key_ to a _value_. The 78SWI-Prolog interface to Berkeley DB allows for fast storage of arbitrary 79Prolog terms including cycles and constraints in the database. 80 81Accessing a database consists of four steps: 82 83 1. Initialise the default DB environment using bdb_init/1 or 84 create an explicit DB environment using bdb_init/2. This 85 step is optional, providing simple non-transactional file access 86 when omitted. 87 2. Open a database using bdb_open/4, returning a handle to the 88 database. 89 3. Accessing the data using bdb_put/3, bdb_get/3, etc. 90 4. Closing a database using bdb_close/1. When omitted, all open 91 databases are closed on program halt (see at_halt/1). 92 93*Errors* reported by the underlying database are mapped to an exception 94of the form error(bdb(Code,Message,Object), _), where `Code` is an atom 95for well known errors and an integer for less known ones. `Message` is 96the return from the db_strerror() function and `Object` is the most 97related Prolog object, typically a database or database environment 98handle. If `Code` is an atom, it is the lowercase version of the 99associated C macro after string the =|DB_|= prefix. Currently the 100following atom-typed codes are defined: `lock_deadlock`, `runrecovery`, 101`notfound`, `keyempty`, `keyexist`, `lock_notgranted` and 102`secondary_bad`. 103*/ 104 105%! bdb_init(+Options) is det. 106%! bdb_init(-Environment, +Options) is det. 107% 108% Initialise a DB _environment_. The predicate bdb_init/1 109% initialises the _default_ environment, while bdb_init/2 creates 110% an explicit environment that can be passed to bdb_open/4 using 111% the environment(+Environment) option. If bdb_init/1 is called, 112% it must be called before the first call to bdb_open/4 that uses 113% the default environment. If bdb_init/1 is not called, the 114% default environment can only handle plain files and does not 115% support multiple threads, locking, crash recovery, etc. 116% 117% Initializing a BDB environment always requires the home(+Dir) 118% option. If the environment contains no databases, the argument 119% create(true) must be supplied as well. 120% 121% The currently supported options are listed below. The name of 122% the boolean options are derived from the DB flags by dropping 123% the =DB_= prefix and using lowercase, e.g. =DB_INIT_LOCK= 124% becomes `init_lock`. For details, please refer to the DB manual. 125% 126% - create(+Bool) 127% If `true`, create any underlying file as required. By 128% default, no new files are created. This option should be 129% set for prograns that create new databases. 130% - failchk(+Bool) 131% - home(+Home) 132% Specify the DB home directory, the directory holding the 133% database files. The directory must exist prior to calling 134% these predicates. 135% - init_lock(+Bool) 136% Enable locking (=DB_INIT_LOCK=). Implied if transactions 137% are used. 138% - init_log(+Bool) 139% Enable logging the DB modifications (=DB_INIT_LOG=). Logging 140% enables recovery of databases in case of system failure. 141% Normally it is used in combination with transactions. 142% - init_mpool(+Bool) 143% Initialize memory pool. Impicit if mp_size(+Size) or 144% mp_mmapsize(+Size) is specified. 145% - init_rep(+Bool) 146% Init database replication. The rest of the replication 147% logic is not yet supported. 148% - init_txn(+Bool) 149% Init transactions. Implies init_log(true). 150% - lockdown(+Bool) 151% - mp_size(+Integer) 152% - mp_mmapsize(+Integer) 153% Control memory pool handling (=DB_INIT_MPOOL=). The 154% `mp_size` option sets the memory-pool used for 155% caching, while the `mp_mmapsize` controls the maximum size 156% of a DB file mapped entirely into memory. 157% - private(+Bool) 158% - recover(+Bool) 159% Perform recovery before opening the database. 160% - recover_fatal(+Bool) 161% Perform fatal recovery before opening the database. 162% - register(+Bool) 163% - server(+Host, [+ServerOptions]) 164% Initialise the DB package for accessing a remote 165% database. Host specifies the name of the machine running 166% `berkeley_db_svc`. Optionally additional options may be 167% specified: 168% - server_timeout(+Seconds) 169% Specify the timeout time the server uses to determine 170% that the client has gone. This implies the server will 171% terminate the connection to this client if this client 172% does not issue any requests for the indicated time. 173% - client_timeout(+Seconds) 174% Specify the time the client waits for the server to 175% handle a request. 176% - system_mem(+Bool) 177% - transactions(+Bool) 178% Enable transactions, providing atomicy of changes and 179% security. Implies logging and locking. See 180% bdb_transaction/1. 181% - thread(+Bool) 182% Make the environment accessible from multiple threads. 183% - thread_count(+Integer) 184% Declare an approximate number of threads in the database 185% environment. See =|DB_ENV->set_thread_count()|=. 186% - use_environ(+Bool) 187% - use_environ_root(+Bool) 188% - config(+ListOfConfig) 189% Specify a list of configuration options, each option is of 190% the form Name(Value). Currently unused. 191 192%! bdb_close_environment(+Environment) is det. 193% 194% Close a database environment that was explicitly created using 195% bdb_init/2. 196 197%! bdb_current_environment(-Environment) is nondet. 198% 199% True when Environment is a currently known environment. 200 201bdb_current_environment(Environment) :- 202 bdb_current_environment_(Environment), 203 bdb_is_open_env(Environment). 204 205bdb_current_environment_(Env) :- 206 ( var(Env) 207 -> ( Env = default 208 ; current_blob(Env, bdb_env) 209 ) 210 ; ( Env == default 211 -> true 212 ; current_blob(Env, bdb_env) 213 ) 214 ). 215 216%! bdb_environment_property(?Environment, ?Property) is nondet. 217% 218% True when Property is a property of Environment. Defined 219% properties are all boolean options defined with bdb_init/2 220% and the following options: 221% 222% - home(-Path) 223% Path is the absolute path name for the directory used 224% as database environment. 225% - open(-Boolean) 226% True if the environment is open. 227 228bdb_environment_property(Env, Property) :- 229 bdb_current_environment_(Env), 230 ( bdb_is_open_env(Env) 231 -> ( var(Property) 232 -> env_property(Property), 233 bdb_env_property_(Env, Property) 234 ; bdb_env_property_(Env, Property) 235 ) 236 ; Property = open(false) 237 ). 238 239bdb_env_property_(Env, home(Home)) :- 240 !, 241 bdb_env_property(Env, home(Home0)), 242 prolog_to_os_filename(Home, Home0). 243bdb_env_property_(Env, Prop) :- 244 bdb_env_property(Env, Prop). 245 246env_property(open(true)). 247env_property(home(_)). 248env_property(init_lock(_)). 249env_property(init_log(_)). 250env_property(init_mpool(_)). 251env_property(init_rep(_)). 252env_property(init_txn(_)). 253env_property(recover(_)). 254env_property(recover_fatal(_)). 255env_property(use_environ(_)). 256env_property(use_environ_root(_)). 257env_property(create(_)). 258env_property(lockdown(_)). 259env_property(failchk(_)). 260env_property(private(_)). 261env_property(register(_)). 262env_property(system_mem(_)). 263env_property(thread(_)). 264 265 266%! bdb_open(+File, +Mode, -DB, +Options) is det. 267% 268% Open File holding a database. Mode is one of `read`, providing 269% read-only access or `update`, providing read/write access. 270% Options is a list of options. Supported options are below. The 271% boolean options are passed as flags to =|DB->open()|=. The 272% option name is derived from the flag name by stripping the 273% =|DB_|= prefix and converting to lower case. Consult the 274% Berkeley DB documentation for details. 275% 276% - auto_commit(+Boolean) 277% Open the database in a transaction. Ensures no database 278% is created in case of failure. 279% - create(+Boolean) 280% Create a new database of the database does not exist. 281% - dup(+Boolean) 282% Do/do not allow for duplicate values on the same key. 283% Default is not to allow for duplicates. 284% - excl(+Boolean) 285% Combined with create(true), fail if the database already 286% exists. 287% - multiversion(+Boolean) 288% Open the database with support for multiversion concurrency 289% control. The flag is passed, but no further support is 290% provided yet. 291% - nommap(+Boolean) 292% Do not map this database into process memory. 293% - rdonly(+Boolean) 294% Open the database for reading only. 295% - read_uncommitted(+Boolean) 296% Read operations on the database may request the return of 297% modified but not yet committed data. This flag must be 298% specified on all DB handles used to perform dirty reads or 299% database updates, otherwise requests for dirty reads may not 300% be honored and the read may block. 301% - thread(+Boolean) 302% Enable access to the database handle from multiple threads. 303% This is default if the corresponding flag is specified for 304% the environment. 305% - truncate(+Boolean) 306% When specified, truncate the underlying file, i.e., start 307% with an empty database. 308% - database(+Name) 309% If File contains multiple databases, address the named 310% database in the file. A DB file can only consist of multiple 311% databases if the bdb_open/4 call that created it specified 312% this argument. Each database in the file has its own 313% characteristics. 314% - environment(+Environment) 315% Specify a database environment created using bdb_init/2. 316% - key(+Type) 317% - value(+Type) 318% Specify the type of the key or value. Allowed values are: 319% - term 320% Key/Value is a Prolog term (default). This type allows for 321% representing arbitrary Prolog data in both keys and value. 322% The representation is space-efficient, but Prolog 323% specific. See PL_record_external() in the SWI-Prolog 324% Reference Manual for details on the representation. The 325% other representations are more neutral. This implies they 326% are more stable and sharing the DB with other languages is 327% feasible. 328% - atom 329% Key/Value is an atom. The text is represented as a 330% UTF-8 string and its length. 331% - c_blob 332% Key/Value is a blob (sequence of bytes). On output, 333% a Prolog string is used. The input is either a Prolog 334% string or an atom holding only characters in the range 335% [0..255]. 336% - c_string 337% Key/Value is an atom. The text is represented as a C 338% 0-terminated UTF-8 string. 339% - c_long 340% Key/Value is an integer. The value is represented as a 341% native C long in machine byte-order. 342% 343% @arg DB is unified with a _blob_ of type `db`. Database handles 344% are subject to atom garbage collection. 345% @error permission_error(access, bdb_environment, Env) if an 346% environment is not thread-enabled and accessed from multiple 347% threads. 348 349%! bdb_close(+DB) is det. 350% 351% Close BerkeleyDB database indicated by DB. DB becomes invalid 352% after this operation. An attempt to access a closed database 353% is detected reliably and results in a permission_error 354% exception. 355 356%! bdb_put(+DB, +Key, +Value) is det. 357% 358% Add a new key-value pair to the database. If the database does 359% not allow for duplicates the possible previous associated with 360% Key is replaced by Value. 361 362%! bdb_del(+DB, ?Key, ?Value) is nondet. 363% 364% Delete the first matching key-value pair from the database. If 365% the database allows for duplicates, this predicate is 366% non-deterministic, otherwise it is _semidet_. The enumeration 367% performed by this predicate is the same as for bdb_get/3. See 368% also bdb_delall/3. 369 370%! bdb_delall(+DB, +Key, ?Value) is det. 371% 372% Delete all matching key-value pairs from the database. With 373% unbound Value the key and all values are removed efficiently. 374 375bdb_delall(DB, Key, Value) :- 376 var(Value), 377 !, 378 bdb_del(DB, Key). % this is much faster 379bdb_delall(DB, Key, Value) :- 380 ( bdb_del(DB, Key, Value), 381 fail 382 ; true 383 ). 384 385%! bdb_get(+DB, ?Key, -Value) is nondet. 386% 387% Query the database. If the database allows for duplicates this 388% predicate is non-deterministic, otherwise it is _semidet_. Note 389% that if Key is a term this matches stored keys that are 390% _variants_ of Key, *not* unification. See =@=/2. Thus, after 391% bdb_put(DB, f(X), 42), we get the following query results: 392% 393% - bdb_get(DB, f(Y), V) binds Value to `42`, while `Y` is left 394% unbound. 395% - bdb_get(DB, f(a), V) _fails_. 396% - bdb_enum(DB, f(a), V) succeeds, but does not perform any 397% indexing, i.e., it enumerates all key-value pairs and 398% performs the unification. 399 400%! bdb_enum(+DB, -Key, -Value) 401% 402% Enumerate the whole database, unifying the key-value pairs to 403% Key and Value. Though this predicate can be used with an 404% instantiated Key to enumerate only the keys unifying with Key, 405% no indexing is used by bdb_enum/3. 406 407%! bdb_getall(+DB, +Key, -Values) is semidet. 408% 409% Get all values associated with Key. Fails if the key does not 410% exist (as bagof/3). 411 412%! bdb_current(?DB) is nondet. 413% 414% True when DB is a handle to a currently open database. 415 416bdb_current(DB) :- 417 current_blob(DB, bdb), 418 bdb_is_open(DB). 419 420%! bdb_closeall is det. 421% 422% Close all currently open databases and environments. This is 423% called automatically after loading this library on process 424% terminatation using at_halt/1. 425 426bdb_closeall :- 427 close_databases, 428 close_environments. 429 430close_databases :- 431 forall(bdb_current(DB), 432 catch(bdb_close(DB), 433 E, 434 print_message(warning, E))). 435 436close_environments :- 437 forall(bdb_current_environment(DB), 438 catch(bdb_close_environment(DB), 439 E, 440 print_message(warning, E))). 441 442terminate_bdb :- 443 ( current_predicate(bdb_init/1) % library was loaded ok 444 -> bdb_closeall 445 ; true 446 ). 447 448:- at_halt(terminate_bdb). 449 450%! bdb_transaction(:Goal) is semidet. 451%! bdb_transaction(+Environment, :Goal) is semidet. 452% 453% Start a transaction, execute Goal and terminate the transaction. 454% Only if Goal succeeds, the transaction is commited. If Goal 455% fails or raises an exception, the transaction is aborted and 456% bdb_transaction/1 either fails or rethrows the exception. Of 457% special interest is the exception 458% 459% == 460% error(package(db, deadlock), _) 461% == 462% 463% This exception indicates a deadlock was raised by one of the DB 464% predicates. Deadlocks may arise if multiple processes or threads 465% access the same keys in a different order. The DB 466% infra-structure causes one of the processes involved in the 467% deadlock to abort its transaction. This process may choose to 468% restart the transaction. 469% 470% For example, a DB application may define `{Goal}` to realise 471% transactions and restart these automatically is a deadlock is 472% raised: 473% 474% == 475% {Goal} :- 476% catch(bdb_transaction(Goal), E, true), 477% ( var(E) 478% -> true 479% ; E = error(package(db, deadlock), _) 480% -> {Goal} 481% ; throw(E) 482% ). 483% == 484% 485% @arg Environment defines the environment to which the 486% transaction applies. If omitted, the default environment 487% is used. See bdb_init/1 and bdb_init/2. 488 489%! bdb_version(-Version:integer) is det. 490% 491% True when Version identifies the database version. Version 492% is an integer defined as: 493% 494% == 495% DB_VERSION_MAJOR*10000 + 496% DB_VERSION_MINOR*100 + 497% DB_VERSION_PATCH 498% == 499 500 501 /******************************* 502 * MESSAGES * 503 *******************************/ 504 505:- multifile 506 prolog:message/3. 507 508prologmessage(error(bdb(Code, Message, Obj), _)) --> 509 [ 'BDB: Error ~w on ~p: ~w'-[Code, Obj, Message] ]