Command line client Cheat Sheet

This cheat sheet summarizes the command line client capabilities.

$ catmandu help
$ catmandu help convert

Convert

Convert one data format to another optionally provide a Fix script to transform the data

$ catmandu convert MARC to JSON < records.mrc
$ catmandu convert MARC to YAML < records.mrc
$ catmandu convert MARC to JSON --pretty 1 < records.mrc
$ catmandu convert MARC to JSON --fix 'marc_map("245","title");remove_field("record")' < records.mrc
$ catmandu convert MARC to CSV --fix myfixes.fix < records.mrc
$ cat myfixes.fix
marc_map("245","title")
remove_field("record")
$ catmandu convert MARC to CSV --fix myfixes2.fix --var source="Springer" < records.mrc
$ cat myfixes2.fix
add_field("source","{{source}")
marc_map("245","title")
remove_field("record")
$ catmandu convert OAI --url http://biblio.ugent.be/oai --set allFtxt to JSON
$ catmandu convert OAI --url http://biblio.ugent.be/oai --set allFtxt to JSON --fix 'retain("title")'
$ catmandu convert SRU --base http://www.unicat.be/sru --query dna	
$ catmandu convert ArXiv --query 'all:electron'
$ catmandu convert PubMed --term 'hochstenbach'
$ cat test.tt
[%- FOREACH f IN record %]
[% _id %] [% f.shift %][% f.shift %][% f.shift %][% f.join(":") %]
[%- END %]
$ catmandu convert MARC to Template --template `pwd`/test.tt < records.mrc 

Import/Export

Store data in a (noSQL) database and export it out again

$ catmandu import JSON to MongoDB --database_name mydb --bag data < records.json
$ catmandu import MARC to MongoDB --database_name mydb --bag data < records.mrc
$ catmandu import MARC to ElasticSearch --index_name mydb --bag data < records.mrc
$ catmandu import MARC to ElasticSearch --index_name mydb --bag data --fix 'marc_map("245a","title")' < records.mrc

$ catmandu export MongoDB --database_name mydb --bag data to JSON
$ catmandu export MongoDB --database_name mydb --bag data to JSON --fix 'retain("_id")'
$ catmandu export Solr --url http://localhost:8983/solr to JSON
$ catmandu export ElasticSearch --index_name mydb to JSON

Copy

Copy data from one database to another

$ catmandu copy MongoDB --database_name items --bag book to ElasticSearch --index_name items --bag book

Count

Count the number of items in a store

$ catmandu count ElasticSearch --index-name shop --bag products --query 'brand:Acme'

Delete

Delete data from a store

# delete items with matching _id
$ catmandu delete ElasticSearch --index-name items --bag book --id 1234 --id 2345

# delete items matching the query
$ catmandu delete ElasticSearch --index-name items --bag book --query 'title:"My Rabbit"'

# delete all items
$ catmandu delete ElasticSearch --index-name items --bag book

Configuration

$ cat catmandu.yml
---
store:
  test1:
   package: MongoDB
   options:
    database_name: mydb
  test2:
   package: ElasticSearch
   options:
    index_name: mydb
  test3:
   package: Solr
   options:
    url: http://localhost:8983/solr

$ catmandu import JSON to test1 < records.json # Mongo
$ catmandu import MARC to test2 < records.mrc  # ElasticSearch
$ catmandu import YAML to test3 < records.yaml # Solr
$ catmandu export test1 to JSON                # Mongo
$ catmandu export test2 to JSON                # ElasticSearch
$ catmandy export test3                        # Solr
$ cat fixes.txt
marc_map("245a","title");
marc_map("100","author.$append");
join_field("author",";");
marc_map("008_/10-13","language");
$ catmandu import MARC to test2 --fix fixes.txt

Stream

# Add a file to a FileStore
$ catmandu stream /tmp/myfile.txt to File::Simple --root t/data --bag 1234 --id myfile.txt

# Download a file from a FileStore
$ catmandu stream File::Simple --root t/data --bag 1234 --id myfile.txt to /tmp/output.txt