Tables from JSON, YAML, TOML and CSV
In the terminalela --spec tables-from-json-yaml-toml-and-csv
Each reader goes straight from text to a table: json.table, yaml.table, toml.table and csv.table. It is the path an API's answer or an export takes most often, in one call and at a fraction of the memory: json.table reads the text once and keeps only the rows' cells, never the document as maps and lists.
body = $(curl -sS https://shop.example/api/orders)t = json.table($body, at = "data.orders", columns = {"id": "id", "name": "customer.name", "city": "customer.city", "created": "created"}, types = {"created": Type.TIME}, fallback = {"city": "unknown"})
json.table(text, at = path, columns = {name: path}, flatten = false, types = …, fallback = {column: value}), and yaml.table and toml.table the same:
at =is the path to the list of rows, in the path syntax of Nested data:"data.orders", or the list form for a key with a dot. Left out, the document itself is the list. A path that leads nowhere is an error naming the step that failed, astree.get's is, and one that leads to a map or a plain value an error that says so; a document that is a map is an error suggesting its first list,at = "data.orders". A row is a map, and any other row is an error naming it.- Without
columns =, every key of row 0 is a column, in its order, and every row has the same keys: one more or one fewer is an error naming the row and the key, unlessfallback =names that column's value. A key that holds a map or a list is an error naming the key and showingcolumns =with its parts,columns = {"customer.name": "customer.name", "customer.city": "customer.city"}. flatten = truemakes a nested map's keys columns of their own, named by their paths:customer.name,customer.address.city. At most 256 columns: more is an error giving the count and pointing tocolumns =. A list inside a row stays an error, pointing totree.pick, or a table of its own withat =.flattenandcolumnsare not given together.columns = {name: path}makes a column of each path, in the order written, astree.pickdoes: a row the path doesn't reach is an error naming the row, the path and the step, unlessfallback =names that column's value, which stands in fornulltoo. A path that reaches a map or a list is an error showing the paths into it.- A column takes the type of its values,
nullallowed in any; a column whose values differ in type is an error naming the row, the column and both types, pointing totypes =.types =takestable.of's form,{"id": Type.NUMBER, "created": Type.TIME}, and converts each value: the text"12"to a number, ISO 8601 text to a time, a number or a bool to its text forType.STRING. A value that doesn't convert is an error at its row. - A key in
fallbackthat is no column, anat =or a column's path written wrong, andflattenwithcolumnsare mistakes, whichtrynever catches; every other error is the data's, whichtrycatches. - A fault in the text is said first, at its line and column, as
decodesays it, wherever it is.
t = json.table($body, at = "data.orders")# error: json.table: row 0's key "customer" holds a map, and a cell is text, a number, a bool or null: name its parts as columns, columns = {"customer.name": "customer.name", "customer.city": "customer.city"}, or give flatten = truet = json.table($body, at = "data.orders", flatten = true)$t.columns # ["id", "created", "customer.name", "customer.city"]
csv.table(text, header = true, sep = ",", columns = null, types = null, layouts = null) is table.of(csv.decode(text, sep = sep), …) in one call, with table.of's arguments: the first row names the columns unless header = false, columns = [...] names or renames them, every field is text until types = gives its column a type, and layouts = a time column's layout. A CSV file is already rows, so it takes no at =.
t = csv.table(file.read("orders.csv"), sep = ";", types = {"amount": Type.NUMBER})
json.tablebuilds no tree and no row: what lies off the path to the rows is read and passed over, and each value in a row goes straight into its column. 200,000 orders, 45 MB of JSON, make a table of five columns in 0.17 s at 61 MB at the peak, the text itself 45 MB, wherejson.decode,tree.pickandtable.oftake 0.68 s and 675 MB. A number too large for ela off the path is passed over with the rest.yaml.tablereads the text the same way, through amber-yaml's events, an alias and a merge key asyaml.decodebuilds them;toml.tabletakes the rows of an array of tables,[[orders]], the same way, through amber-toml's. 160,000 orders, 20 MB of YAML, make a table of six columns in 0.54 s at 44 MB, where it took 0.68 s and 303 MB; 20 MB of TOML in 0.17 s at 44 MB, where it took 0.48 s and 498 MB. Rows TOML gives in another shape (an inline array of tables, a row with a dotted key or a table of its own) are read from the decoded document, as before.