Reading and writing are now handled by the yyjson C library, bundled with the
package, rather than by an R JSON package. The practical effects:
0, and larger values could lose their final
digits. Values are now written at the shortest precision that reads back
identically, and parsed to a double in C rather than through R. Files written
by earlier versions are unaffected - the loss happened on read, so re-reading
them with this version recovers the original values (#97).yyjsonr and jsonvalidate are no longer required. Imports is now just
hms; jsonvalidate moved to Suggests and is needed only for the
validate_*() functions (#70).read_dataset_ndjson(), write_dataset_ndjson() and
validate_dataset_ndjson() support the Dataset NDJSON representation, where
the dataset metadata forms the first line and each row follows as a JSON
array.read_dataset_dsjc(), write_dataset_dsjc() and validate_dataset_dsjc()
support Dataset JSON Compressed (.dsjc), a zLib stream of Dataset NDJSON
content. Rows are compressed as they are written, so a large dataset never
needs its uncompressed form held in memory (#78).datasetjson_example() now ships dm.json, dm.ndjson and dm.dsjc.decimals_as_floats has been removed from read_dataset_json(). Columns with
dataType = "decimal" and targetDataType = "decimal" are always converted to
numeric on read, as the Dataset JSON v1.1 specification requires (#77).digits is deprecated in write_dataset_json() and write_dataset_ndjson().
It is ignored, and supplying it warns. Decimals are written at whatever
precision reads back as the same value, so there is no precision left for it
to control. To render values at a fixed precision, format the column to
character yourself and declare it as decimal/decimal in the column
metadata.float_as_decimals is no longer needed to protect precision and setting it
now warns to say so. It remains available as an interoperability choice for
systems that require the decimal type.name, or whose records value is
missing, now reports the problem rather than failing obscurely or silently
substituting a value.read_dataset_json() sets format.sas from the displayFormat column
attribute (#87).write_dataset_json() with float_as_decimals = TRUE wrote NA as a padded
string rather than JSON null (#76).extract_xpt_meta() in the converting_files vignette could return -Inf
for the length of an empty or all-NA string column; the minimum is now 1
(#71).This release provides a significant overhaul of the package due to the updates for Dataset JSON 1.1.0. Performance has also been significantly improved, as well as the main object interface.
pretty parameter on write_dataset_json() to false (#20)Initial development version of datasetjson, introducing core objects, readers and writers.