All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Types of changes
Added
for new features.Changed
for changes in existing functionality.Deprecated
for soon-to-be removed features.Removed
for now removed features.Fixed
for any bug fixes.Security
in case of vulnerabilities.
Added
possibility to filter columns viaselect
property in ingress descriptorsAdded
commandsset-parent-select
andset-child-select
tolino id
Added
import: no
option for columns intables.yaml
configurationAdded
export: presence
option for columns intables.yaml
configurationFixed
issue withimport: file
when data is null, will not fail anymore
Fixed
invalid length for DATE or NUMBER bind variable, bumping oracle driver to 2.8.19
Fixed
hidden global internal timeout of 240s with Oracle driver is now disabled by default
Changed
table or column names were not correctly matched in terms of case sensitivity (lowercase or uppercase) in table.yamlAdded
new commandquery
to execute any SQL query on any SQL datasource
Added
flag--watch
(short-w
) tolino push
command
Fixed
panic during push on Oracle database with anull
column valueFixed
issue with SQL queries involving limit orders in SQL Server databaseFixed
panic when using autotruncate on a null value
Added
columns information and export type using thelino table extract
command, columns and keys organized according to the database orderAdded
flag--only-tables
tolino table extract
command. This flag allows for the extraction of table information exclusively, excluding columns. It has been included to maintain the previous behaviorAdded
flag--with-db-infos
tolino table extract
command. This flag enables the extraction of information regarding column types, length, size, and precision if the column has been configured with these specificationsAdded
flag--autotruncate
tolino push
command. This flag will enable a truncate on each value based eachdbinfo
.length
parameters set in the table.yaml file for each columnsAdded
propertydbinfo
.bytes
to column definition in table.yaml file. Set it to true to truncate the value based on a maximum number of bytes and not characters (assuming utf-8 encoding for now)Added
flags--max-length
and--bytes
tolino table add-column
command. Use it to edit the propertiesdbinfo
.length
anddbinfo
.bytes
of the table.yaml file
Fixed
performance issue withimport: file
Added
sqlserver
supportAdded
flags tolino analyse
:sample-size
,distinct
,limit
,where
andtable
(by default distinct values are not counted)Added
flags tolino analyse
:exlude
andexlude-pk
to exclude columns from analyseAdded
flags tolino pull
:-X
or--exclude-from-file
to filter out lines from a JSON lines file referenceAdded
flags tolino push
:--savepoint
to write primary keys of all commited line to a fileAdded
new import types totable.yaml
:blob
will optimize push of blob column into database,file
will read content of a file at the specified path and import as a blob
Added
commandanalyse
to extract metrics from the database in YAML format
Added
go-ora driver for oracle in replacement of old driver (remove technical prerequisite to install Oracle Instant Client)
Added
flag--pk-translation
allow update of primary keys, by giving a cache.jsonl file containing old and new values for a specific keyAdded
if a JSON object contains a__usingpk__
with a dictionary of key/values, it will be used by the push command to select the target record to update (push update only)Added
where clauses (child and parent) in ingress descriptor to enable non-start table filteringFixed
oracle connector disable/enable contraints cascade
Added
import property intables.yaml
allow to specify format of data to read from JSON stream (string
(default),numeric
,binary
orbase64
(same),datetime
,timestamp
) thereforeimport
andexport
now mirror each other (import
is used at push andexport
at pull) butimport
still allow to specify the data type to pass to database driver (backward compatibility)Added
import property intables.yaml
allow to specify a format AND a type at the same time with theformat(type)
syntax (e.g.import: binary(int64)
)Added
websocket connector with basic auth, supported schemes : ws, wss (BETA)Added
flag--stats
to generate a stat file or HTTP POSTAdded
flag--statsTemplate
to control the format of generated statsFixed
lino push truncate
with--table
option doesn't truncate table #123
Added
export mode all intables.yaml
to export all columns even if some columns are defined in the columns property
Fixed
Bad SQL update statement for oracleFixed
Reset statement after error during push #54Fixed
Continue to close others row writers after error
Changed
order of keys in output JSON lines will be alphabetical when pulling (without configuration in tables.yaml)Added
configuration of export format / import type for columns in tables.yaml, see issue #33 for more informationAdded
MariaDB/MySQL support (thanks to @joaking85)Added
auto-select columns required by a relation but not exported in tables.yamlAdded
new commands to configure tables : add-column and remove-columnAdded
New command to count lines in tablesFixed
limit keyword on DB2 dialectFixed
Push truncate respect child/parent constraint orderFixed
Push truncate will trigger only for attainable tables in the ingress descriptor - tables that are cut out will not be truncatedFixed
push insert with mysql/mariadb connector now works properly with MySQL databaseFixed
push insert with mysql/mariadb connector will not update record if it exists
Added
HTTP connector will now close/reopen request when commit size is reached
Fixed
charset on Content-Type header when pushing to HTTP connector backend
Fixed
some HTTP library doesn't support a body payload on a HTTP GET request, therefore the HTTP connector will pass the "filter" parameter through the headers in addition to the request body
Added
new verb to extract, to get status and to update sequences
Added
new parameter to pull only distinct values from the start table
Added
new datasource type with string connectionhttp://...
LINO can pull/push data to an HTTP Endpoint API
Added
option to change ingress-descriptor filenameChanged
update debian image to last stable (debian:stable-20210816-slim)
Added
update Pimo to v1.8.0
Added
statistics report for push and pull executions (thanks to @CapKicklee)Changed
some info level logs to debug level in pull module
Fixed
Revert convert JSON date to Oracle date format as a workaround for godror
Added
flag to enable or disable coloring in output logs (--color [yes|no|auto])Added
update Pimo to v1.6.1
Fixed
Remove ENTRYPOINT and change CMD in oracle docker image
Added
structured logs (debug & json format) (thanks to @CapKicklee)
Fixed
extract composite primary keys for oracleFixed
protect columns names in insert statement
Fixed
Missing where keyword for Oracle SQL Query
Added
--where flag to use a raw sql where clause to filter rows of start tableAdded
Oracle database supportSecurity
remove connection string from log
Added
First public version released