This repo contains examples of creating dynamic modules for Emacs.
This repo mostly contains proofs of concept so many of these may contain only a single or small number of functions.
- ./examples.org has examples of using these libraries. Usually these have been blogged at some point.
- ./gsl-constants.c shows how to expose constants to Emacs
- ./gsl-linalg.c shows how to deal with arrays and return a vector.
- ./gsl-integration.c shows how to integrate a function. It is a little improved over the blogpost.
- ./gsl-roots.c is similar to the integration library but implements a root finder
./zeromq/ There are two sets of bindings to Zeromq, one is a dynamic module, and one uses emacs-ffi. The ffi was much easier to write, but I anticipate trouble with special types down the road.
./mongodb/ Here I explored an emacs-ffi approach. I ran into trouble dealing with the custom struct types.
I think that a full dynamic module might be required to support this.
https://github.com/anse1/emacs-libpq An Emacs 25 module for accessing PostgreSQL via the libpq client library.
- ./mod-types.c shows how to test the type of an arg, and how to get a function to work for integers and floats. Also this has a little variadic function definition.
- ./mod-list-vec.c contains some test functions for making vectors and indexing them.
- ./emacs-module-helpers.c provides convenience functions like provide and DEFUN.
There are so many things that require boilerplate code. I started ./emacs-module-helpers.c to help with this.
- extract_double/int from an Emacs arg
- defconst/i Define a float/integer constant from a #define in a header
- DEFUN macro to simplify declaring functions to emacs
- provide function
TODO: There should a MACRO for the emacs_value standard functions to send stuff back to Emacs if that is feasible. Maybe something like
CDEFUN fname {}
Not sure that is feasible though.
I have used two designs so far.
The one I use the most returns an emacs value to the emacs environment. These functions have this signature
static emacs_value Fname (emacs_env *env, ptrdiff_t nargs, emacs_value args[], void *data)In this, nargs is an integer of how many arguments were passed from Emacs. The args argument is an array of emacs_value types sent from Emacs. The best I can tell you can use nargs to allow optional arguments. I don’t know what *data is for yet.
This function would be declared to Emacs with:
DEFUN("emacs-func-name", Fname, minnargs, maxnargs,
docstring,
data_pointer);If maxnargs is -2, the function is declared to be variadic.
The second design I have used modifies the Emacs env by making funcalls to emacs commands.
These functions are not registered, but are called in the emacs_init_module code to provide features, like this one:
void provide (emacs_env *env, const char *feature)
{
emacs_value Qfeat = env->intern (env, feature);
emacs_value Qprovide = env->intern (env, "provide");
emacs_value args[] = { Qfeat };
env->funcall (env, Qprovide, 1, args);
}Or to define constants. These functions also are used for binding functions to the environment also.
It is moderately tedious to test modules because once you load them you cannot rebuild and reload them. You have to kill emacs and try again. Here is a pretty fast way to test modules. You develop in one emacs, and run the appropriate make command to build it.
Add test functions to ./tests.el. Then run them from the command line like this. Right now most of these functions just test that no segfaults occur and that there is some output. It would be better to integrate ert-testing…
emacs -batch -q -l tests.el -f test-linalgOr add lines like that one to the ./Makefile
For my own notes here are all the resources on dynamic modules I know of:
Here are the official Emacs header and example: emacs-module.h: http://git.savannah.gnu.org/cgit/emacs.git/tree/src/emacs-module.h?id=e18ee60b02d08b2f075903005798d3d6064dc013 mod_test.c: http://git.savannah.gnu.org/cgit/emacs.git/tree/modules/mod-test/mod-test.c?id=e18ee60b02d08b2f075903005798d3d6064dc013
This simple example in C http://diobla.info/blog-archive/modules-tut.html
- joymacs
- http://nullprogram.com/blog/2016/11/05/
- mruby
- https://github.com/syohex/emacs-mruby-test
- https://github.com/tromey/emacs-ffi
- an actual ffi for emacs
- elfuse
- https://github.com/vkazanov/elfuse a file system in Emacs
- asynchronous events
- http://nullprogram.com/blog/2017/02/14/ related to elfuse
- emacs-sqlite3
- sqlite3 binding of Emacs Lisp
- emacs-parson
- JSON parser with dynamic module feature with parson
- libyaml
- libyaml
- emacs-perl
- Embed Perl into Emacs
- https://github.com/syohex/emacs-eject
- eject a cd
- emacs-capstone
- elisp bindings for the capstone disassembler
- emacs-csound
- EmacsLisp link to Csound’s API via Emacs Modules
- emacs-cmigemo
- Emacs dynamic module for cmigemo
- emacs-cipher
- OpenSSL cipher binding of Emacs Lisp
- emacs-lua
- Lua engine from Emacs Lisp
- emacs-ztd
- libzstd binding of Emacs Lisp
- mem-cached
- libmemcached
- https://coldnew.github.io/2d16cc25/
- in Chinese, but with code
A collection of module resources: https://github.com/emacs-pe/emacs-modules
- Nim https://github.com/yuutayamada/nim-emacs-module
- OCaml https://github.com/janestreet/ecaml
- Rust https://github.com/lunaryorn/emacs-module.rs https://github.com/jjpe/emacs_module_bindings
- golang
- https://github.com/sigma/go-emacs writing modules in go
This may not be a dynamic module but claims an ffi haskell https://github.com/knupfer/haskell-emacs
It seems like the path forward on this is probably to implement the C functions as close to the documentation as possible and then to create elisp wrappers that do resource management, e.g. freeing memory, destroying pointers, etc.
This would enable me to use the DEFINE’s most easily I think.
The big downside of dynamic modules so far is how to get decent docstrings and signatures.
./zeromq/ ./zeromq/zeromq.org - contains a dynamic module and an ffi implementation.
./mongodb/ ffi: ./mongodb/mongo-ffi.org and ./mongodb/bson.org
My intuition is that a dynamic module is better than the ffi, at least until it is obvious how to handle custom data types and structs in the ffi.
There is already one interface for this. I don’t know how it maps to SQL queries, but it might enable cleaner integration to full-text search. If one was dreaming, there might also be an interface to something like ElasticSearch.
Similar to the BSON/mongoc situation it seems likely that a full dynamic module is necessary here too because of ffi limitations on custom structs.
I think getting a decent linear algebra capability with minimal broadcasting would be helpful. We can use vectors and have a generalized dot product.
It might be best to see if I can wrap Numpy for this.
(dot [1 2] [3 4]) ; a scalar
(dot [[1 1]
[2 2]]
[1 1]) ; here b is implied as a columnIt would be helpful to have a vector-1d-p function, and a vector size function. The vector size function should probably work recursively so you can pass an arbitrary shaped array in.
(defun vector-shape (vec)
"Return a vector of the shape of a vector."
(let ((shape (vector (length vec))))
(if (vectorp (aref vec 0))
(vconcat shape (vector-shape (aref vec 0)))
shape)))
(defun vector-ndims (vec)
"Returns the number of dimensions in a vector."
(length (vector-shape vec)))
(defun vector-numel (vec)
"Returns the number of elements in a vector."
(if (> (length vec) 0)
(seq-reduce '* (vector-shape vec) 1)
0))
(vector-shape [[[1 2][4 5]]])
(vector-ndims [])
(vector-numel [[1 2][3 4]])
Broadcasting rules.
See Adding linear algebra to Emacs with the GSL and dynamic modules
The lengths of the vectors must be the same. Then, we have to make 2d arrays
(rows, cols) (1, m1) and (m2, 1) for the standard matrix multiplication codes.
Assuming the 2d matrix has the shape (m1, n1), then the 1d must have n1 elements, and be converted to a (n1, 1) shape array.
Assuming the 2d matrix has the shape (m1, n1) then the 1d must have m1 elements and we have to make a (1, m1) array out of it for the multiplication.