This program is based on the great work from PatHammer and timOoblik.
At the core is their mouse.c program tweaked Here is the original readme from kinect.mouse

Start.

Kinect Mouse
Credit to:
Tim Flaman - tim@timflaman.com for mouse movement http://www.twitter.com/timOoblik
Robert Walter - for finger tracking and surface touching http://twitter.com/robbeofficial

This is a userspace driver to allow multitouch contol by Microsoft's Kinect sensor.
Required:
openkinect libfreenect drivers - https://github.com/OpenKinect/libfreenect
openCV
OpenGL
Glut
Pthreads
Utouch & Utouch-evemu - https://launchpad.net/utouch-evemu

End.

And Now for 2019/2020 update.

All of this has been done on Raspberry 4b Debian Buster
It will very likely work on Debian and Ubuntu without much changes.
Raspberry 2/3 are not powerful enough.
Warning: Use original raspberry 4 firmware. With the updated firmware version usb communications were disrupted for me and the program did not work anymore
If you do experiment and it works with updated firmware please let me know as I downgraded since.

Installation:

Build OpenKinect
Follow instructions on http://blog.bitcollectors.com/adam/2016/01/kinect-support-for-raspberry-pi-using-libfreenect/

Be sure to add the kinect in the /etc/udev/rules.d/99-kinect.rules as specifed in one of the notes to the post:

sudo vi /etc/udev/rules.d/99-kinect.rules
add:
# ATTR{product}==”Xbox NUI Audio”
SUBSYSTEM==”usb”, ATTR{idVendor}==”045e”, ATTR{idProduct}==”02ad”, MODE=”0666″
# ATTR{product}==”Xbox NUI Camera”
SUBSYSTEM==”usb”, ATTR{idVendor}==”045e”, ATTR{idProduct}==”02ae”, MODE=”0666″
# ATTR{product}==”Xbox NUI Motor”
SUBSYSTEM==”usb”, ATTR{idVendor}==”045e”, ATTR{idProduct}==”02b0″, MODE=”0666″

and then execute
sudo /etc/init.d/udev restart
or
sudo shutdown -r now

Optional : If you want to reenable the two mics inside the Kinect (Why not after all they are there...) follow this instructions:

https://pierre.porcheret.org/index.php?p=YmxvZw==&article=457

And that is it for Prerequisites.

Compiling:
git clone this repo 
goto the build directory
give a make command 
that's it.

If as it should it compiled without errors copy the kmouse_mm binary file to your bin path to have it accessible anywhere.
Dont't forget to make the compiled binary executable (chmod +x)

The command accepts a veeery long list of parameters to tweak the output for your setup and debug:

- NearPixel_TooClose: Number of maximum near pixel to accept before sending a too close message
- NearPixel_TooFarOrNoise: Number of minimum near pixel to accept as pointer (i.e. not noise)
- KinectLogLevel: Log level to output (0-7) 0 = nothing / 0 Flood
- KinectSwhowScreen: 0: no output 1: Show camera and depth camera
- KinectAngle: Start angle for kinect -30 : 30
- KinectLed: Led color to light on 0-6
- ClickSize: Size of the area to be considered as mouse steady for click
- ClickPauseCount: number of subsequent frames with steady mouse to trigger a click 
- minimum_stroke_points: minimum number of points to evaluate as a stroke
- maximum_stroke_points: maximum number of points to evaluate as a stroke
- Horizontal Variance threshold: maximum variance of coords in horizontal direction for a set of coords to be considered a vertical swipe
- Vertical Variance threshold: maximum variance of coords in vertical direction for a set of coords to be considered an horizontal swipe
- near_threshold: depth for near points
- far_threshold: depth for far points
- Elbow X
- ElbowY
- JSon Output X
- Output program log
- Output sensor status
- Output Click events
- Output Coordinates events
- Output Swipe events
- Verbose debug to stdout
- Pause Verbose debug output at swipe evaluation

It's easy to miss one parameter : in the git you find a couple of pretyped commands (only....sh shell scripts)

How does it work:
The original virtual mouse is working by assuming you will be pointing your hand towards the kinect.
Hence your hand will be the nearest object.
The program locates your hand and scales it's position to the surface of the screen.
Simple but effective enough.
A mouse click is assumed when the pointer is steady in a square area of pixels for more then n subsequent frames.

New features added/modified (documentation WIP)
recognize swipes
ouput json