Changelog
Source:NEWS.md
nodbi 0.11.0
CRAN release: 2024-11-10
Potentially breaking changes
-
docdb_query()
modified so that it returns a data frame, in which each column has just one type (atomic or list) across all the rows of the respective column (previously, e.g. a mix of single-item lists simplified to atomic values and of multi-item lists were returned)
Changes
-
docdb_create()
anddocdb_update()
for SQLite and PostgreSQL (only if on localhost) now import directly and fast fromndjson
files, in analogy to DuckDB (needs RSQLite >= 2.3.7.9014) - Refactored
docdb_update()
forsrc_couchdb()
- Add message from
docdb_create()
if a data frame has column names with a dot(s) since dots innodbi
are used forJSON
dot paths - Add code to check database backend version requirements
- Adding info if PostgreSQL database is not yet created
- Factored out further code
nodbi 0.10.7
CRAN release: 2024-09-29
Changes
- uses new features of
duckdb
1.11.0 for refactoring ofdocdb_query()
, accelerating queries - accelerated creating and updating from file
nodbi 0.10.6
CRAN release: 2024-07-25
Changes
- partial refactoring of
docdb_query()
, accelerating queries up to 20-fold for SQLite, DuckDB, and acceleratinglistfields = TRUE
several times for DuckDB
Bug fix
- address
docdb_query()
not working for cases when dot paths had no counts between fields - address wrong database size printing
nodbi 0.10.5
CRAN release: 2024-06-30
Bug fix
- stop if query is invalid even though JSON is valid
- print information also for MongoDB connection object
Changes
- code cleaning, parameters checking
- document that
$regex
indocdb_query()
is case-sensitive
nodbi 0.10.2
CRAN release: 2024-02-24
nodbi 0.10.1
CRAN release: 2024-02-04
Bug fix
- make
docdb_get()
work again forsrc_sqlite()
by castingJSONB
back toJSON
nodbi 0.10.0
CRAN release: 2024-01-21
Deprecated
- empty parameter
query
now triggers a warning as it should be a valid JSON string; changequery = ""
intoquery = "{}"
Changes
- adapted to use new, faster
JSONB
functions inSQLite
3.45.0 (RSQLite
>= 2.3.4.9005) - refactored parts of
docdb_create()
to speed up handling large data frames and lists - made Elasticsearch to immediately refresh index after
docdb_create()
and other functions -
docdb_update()
now reports which records failed to update and then continues -
docdb_delete()
now returns harmonised success logical value across backends
Potentially breaking change
docdb_query()
reimplementation to have the same functionality across all databases (DuckDB, SQLite, PostgreSQL, MongoDB, Elasticsearch, CouchDB); even though the API and unit tests remained, user provisions may break e.g. to handle return values of databases that previously were incompletely implemented (in particular Elasticsearch and CouchDB). Details:
-
query
can now be complex (nested, various operators) and is digested with a Javascript helper -
fields
can now be nested fields (e.g.,friends.name
) to directly return values lifted from the nested field -
listfields
parameter newly implemented to return dot paths for all fields of all or selected documents in collection - expanded use of
jq
viajqr
for mangling parameters, selecting documents, filtering fields and lifting nested field values - if no data are found, returns
NULL
(previously some backends returned an empty data frame) -
docdb_query(src, key, query = "{}", fields = "{}")
now delegates todocdb_get(src, key)
-
_id
is always returned, unless specified with"_id": 0
in parameterfields
- for
scr_postgres
, only fewer than 50 fields if any can be specified infields
- for
src_sqlite
, minimise the use of the time-costlyjson_tree
- workaround for path collisions of MongoDB
- some acceleration of
docdb_query()
- factored out common code
- expanded testing
- updated docs
nodbi 0.9.7
CRAN release: 2023-09-17
Changes
- changed
docdb_update()
to directly use NDJSON from file for duckdb - cleaned up unnecessary code in
docdb_create()
- no more using transactions with
src_duckdb()
Bug fixes
- regression error from not specifying top-level jq script
- corrected and improve field selection in
docdb_query()
- corrected test exceptions for mongodb, updated GitHub Actions, expanded tests
nodbi 0.9.6
CRAN release: 2023-08-07
Bug fixes
- corrected marginal case in
docdb_query.src_duckdb()
- corrected minimum R version
- replaced in tests
httpbin
withwebfakes
- removed explicit UTF-8 encoding reference
- speed up in
docdb_query()
- switched to v2 GitHub r-lib/action for R CMD check
nodbi 0.9.3
CRAN release: 2023-04-23
Changes
-
docdb_update()
now can do bulk updates when _id’s are invalue
(for SQLite, DuckDB, PostgreSQL, MongoDB; not yet for CouchDB and Elastic)
nodbi 0.9.2
CRAN release: 2023-03-26
Bug fixes
-
src_duckdb()
handles when json_type returns NULL for non-existing path -
src_sqlite()
handles when text includes double quotation marks
Changes
- added warning if DuckDB’s JSON extension is not available; improve instructions; see also issue #45
- minor simplification of
docdb_exists()
forsrc_mongo()
, and ofdocdb_query()
for SQL databases
nodbi 0.9.1
CRAN release: 2022-11-20
Changes
- corrected closing connections to SQL database backends upon session restart
- improved provisions for parallel write access and corresponding tests
- capture marginal case of no rows in
docdb_query()
nodbi 0.7.1
CRAN release: 2022-05-04
Bug fixes
- fixed
docdb_query()
to account for change in SQLite 3.38.3 adding quotation of labels (closes issue #44), test added - made
docdb_query()
work for PostgreSQL when a string used with the$in
operator has a comma(s), test added
nodbi 0.7.0
CRAN release: 2022-01-04
Improvements
-
docdb_create()
now supports file names and http urls as argumentvalue
for importing data -
docdb_create()
(and thusdocdb_update()
) now supports quantifiers (e.g., ‘[a-z]{2,3}’) in regular expressions
Bug fixes
- for SQLite, return
FALSE
like other backends when usingdocdb_delete()
for a non-existing container (table, in the case of SQLite) - better handle special characters and encodings under Windows
nodbi 0.5.0
CRAN release: 2021-11-21
Improvements
- identical API for
docdb_*()
functions so thatquery
andfields
parameters can be used across database backends - identical return values across database backends
- re-factored recently added functions for RSQLite
- re-factored most functions to provide identical API
- performance (timing and memory use) profiled and optimised as far as possible
- testing now uses the same test file across databases
- currently, no more support for redis (no way was found to query and update specific documents in a container)
-
docdb_list()
added as function to list container in database
nodbi 0.4.4
Improvements
- made remaining
docdb_*()
functions return a logical indicating the success of the function (docdb_create
,docdb_delete
), or a data frame (docdb_get
,docdb_query
), or the number of documents affected by the function (docdb_update
) - change testing approach
Bug fixes
-
docdb_get()
to not return ’_id’ field forsrc_{sqlite,mongo}
since already used for row names
nodbi 0.4.3
CRAN release: 2021-07-23
Improvements
-
docdb_query.src_sqlite()
now handles JSON objects, returning nested lists (#40) -
src_sqlite()
now uses transactions for relevant functions (#39) -
docdb_update.src_mongo()
now returns the number of upserted or matched documents, irrespective of whether they were updated or not
Bug fixes
-
docdb_get()
to not return ’_id’ field forsrc_{sqlite,mongo}
since already used for row names
nodbi 0.4.2
CRAN release: 2021-01-14
Bug fixes
- fix for
src_couchdb()
: we were not setting user and password correctly internally, was causing issues in CouchDB v3 (#35) thanks to @drtagkim for the pull request
nodbi 0.4.0
CRAN release: 2019-11-11
Improvements
- in
docdb_query()
anddocdb_get()
, for sqlite source, use a connection instead of a regular file path to avoid certain errors on Windows (#33) work by @rfhb - in
docdb_query()
anddocdb_create()
for sqlite source, fix to handle mixed values of different types (#34) work by @rfhb - some Sys.sleep’s added to Elasticserch eg’s to make sure data is available after creation, and before a data request
nodbi 0.3.0
CRAN release: 2019-09-18
New features
- new author Ralf Herold, with contribution of new functions for working with SQLite/json1. new functions:
src_sqlite
,print.src_sqlite
,docdb_create.src_sqlite
,docdb_delete.src_sqlite
,docdb_exists.src_sqlite
,docdb_get.src_sqlite
,docdb_query.src_sqlite
, anddocdb_update.src_sqlite
. includes new datasetcontacts
(#25) (#27) (#28) (#29) (#30) (#31) -
docdb_update
gains method for working with MongoDB, via (#27)
Improvements
- added
.github
files in the source repository to facilitate contributions -
src_mongo
changes, improved behavior, via (#27)
Defunct
-
etcd
(via theetseed
package) integration has been removed from this package as etcd doesn’t really fit the main goal of the pkg. functions now defunct are:src_etcd
,docdb_create.src_etcd
,docdb_delete.src_etcd
,docdb_exists.src_etcd
,docdb_get.src_etcd
, andprint.src_etcd
(#26)
nodbi 0.2.0
CRAN release: 2019-04-13
New features
-
docdb_get()
gainslimit
parameter to do pagination, for CouchDB, Elasticsearch and MongoDB only (#17) (#23) - gains function
docdb_query()
to send queries to each backend (#18) (#22) - gains function
docdb_exists()
to check if a database or equivalent exists (#21) (#22)
Improvements
- Updated package for new version of
elastic
, which has slightly different setup for connecting to the Elasticsearch instance (#20)