Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Config as YAML files instead of python. Live translation of logs and TTS #31

Merged
merged 6 commits into from
Jul 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ To use the Googla Coral USB Accelerator, first flash the Pi SD card with the ima

(I attempted to install the required software from the coral [getting started guide](https://coral.ai/docs/accelerator/get-started#1-install-the-edge-tpu-runtime) but I was unable to get past an error relating to grpico "GLIBC_2.29' not found")

Alternatively, set Config.VISION_TECH to `opencv` for the original (slower) facial recognition. I am not updating this anymore so you may find some integration issues.
Alternatively, set Config.vision.tech to `opencv` for the original (slower) facial recognition. I am not updating this anymore so you may find some integration issues.

## Installation
```
Expand Down
7 changes: 7 additions & 0 deletions config/buzzer.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
buzzer:
pin: 27
docs:
description: "The buzzer is a piezo speaker that can be used to play tones and melodies."
events:
sub:
- speech
3 changes: 3 additions & 0 deletions config/hotword.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
hotword:
model: ''
sleep: 0.03
11 changes: 11 additions & 0 deletions config/motion.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
motion:
pin: 13
docs:
description: "The motion module is used to detect motion using a microwave sensor."
event:
sub:
- loop:1:
description: "Trigger read evey second."
pub:
- motion:
description: "Sent when motion is detected."
37 changes: 37 additions & 0 deletions config/neopixel.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
neopixel:
pin: None
count: 7
docs:
description: "The neopixel module is used to drive neopixel LEDs."
events:
sub:
- led:
description: "Set specific LEDs"
args:
- identifiers: "array[integer] | integer"
- color: string
- led:full:
description: "Set all LEDs"
args:
- color: string
- led:eye:
description: "Set eye LEDs"
args:
- color: string
- led:off:
description: "Turn off all LEDs"
- led:flashlight:
description: "Turn on all eye LEDs full white"
args:
- on: boolean
- led:party:
description: "Turn on all LEDs in random colors"
- exit:
description: "Exit module."
- speech:
description: "Attempt to process commands from speech (light on | off)"
args:
- msg: string
pub:
- log:
description: "Send messages to logger."
6 changes: 6 additions & 0 deletions config/rfid.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
rfid:
pin: None
cards:
"1234567890": "John Doe"
"0987654321": "Jane Doe"
"1234567891": "John Doe"
38 changes: 38 additions & 0 deletions config/servos.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
servos:
conf:
leg_l_hip: { id: 0, pin: 9, range: [20, 160], start: 19 }
leg_l_knee: { id: 1, pin: 10, range: [5, 175], start: 12 }
leg_l_ankle: { id: 2, pin: 11, range: [40, 180], start: 52 }
leg_r_hip: { id: 3, pin: 6, range: [20, 160], start: 93 }
leg_r_knee: { id: 4, pin: 7, range: [5, 175], start: 190 }
leg_r_ankle: { id: 5, pin: 8, range: [40, 180], start: 88 }
tilt: { id: 6, pin: 2, range: [60, 120], start: 75 }
pan: { id: 7, pin: 3, range: [20, 160], start: 50 }
docs:
description: "The servos module is used to control servos. These are handled by the arduino but positions can be requested by this framework via serial"
events:
sub:
- servo:[identifier]:mvabs:
description: "Move servo to absolute position."
args:
- percentage: integer
- servo:[identifier]:mv:
description: "Move servo to relative position."
args:
- percentage: integer
- safe: boolean
pub:
- log:
description: "Send messages to logger."
- log:error:
description: "Send error messages to logger."
- power:use:
description: "Request power usage."
- power:release:
description: "Release power usage."
- serial:
description: "Send serial data to arduino."
args:
- type: string
- identifier: integer
- message: string
18 changes: 18 additions & 0 deletions config/translator.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
translator:
default:
src: "en"
dest: "es"
docs:
description: "The translator module is used to translate any text from one language to another"
methods:
- init:
description: "Initialize the translator module"
args:
- src: string
- dest: string
- request:
description: "Translate text from one language to another. Languages can be specified or use defaults"
args:
- text: string
- src: string
- dest: string
29 changes: 29 additions & 0 deletions config/vision.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
vision:
tech: "coral"
initial_mode: "object"
modes: ["object", "face"]
debug: false
docs:
description: "The vision module is used to detect objects and faces."
events:
sub:
- exit:
description: "Exit module."
pub:
- log:
description: "Send messages to logger."
- vision:detect:object:
description: "The vision module has detected an object."
args:
- name: string
- vision:detect:face:
description: "The vision module has detected a face."
args:
- name: string
- vision:matches:
description: "List of vision matches."
args:
- matches: array
- labels: array
- vision:nomatch:
description: "No matches."
45 changes: 25 additions & 20 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,15 @@
from modules.buzzer import Buzzer
from modules.pitemperature import PiTemperature

if Config.VISION_TECH is 'opencv':
from modules.translator import Translator

if Config.get('vision', 'tech') is 'opencv':
from modules.opencv.vision import Vision
from modules.opencv.tracking import Tracking
from modules.opencv.train_model import TrainModel
from modules.opencv.video_stream import VideoStream
from modules.opencv.timelapse import Timelapse
elif Config.VISION_TECH is 'coral':
elif Config.get('vision', 'tech') is 'coral':
from modules.coral.vision import Vision
from modules.coral.tracking import Tracking

Expand All @@ -62,7 +64,9 @@ def main():
print('Starting...')

path = os.path.dirname(__file__)
log = LogWrapper(path=os.path.dirname(__file__))
translator = Translator(src=Config.get('translator', 'default')['src'], dest=Config.get('translator', 'default')['dest'])
log = LogWrapper(path=os.path.dirname(__file__), translator=translator)


# Throw exception to safely exit script when terminated
signal.signal(signal.SIGTERM, Config.exit)
Expand All @@ -74,8 +78,9 @@ def main():
serial = ArduinoSerial()

servos = dict()
for key in Config.servos:
s = Config.servos[key]
servo_conf = Config.get('servos','conf')
for key in servo_conf:
s = servo_conf[key]
servos[key] = Servo(s['pin'], key, s['range'], s['id'], start_pos=s['start'])

# pub.sendMessage('log', msg="[Main] Starting pan test")
Expand All @@ -98,11 +103,11 @@ def main():
#return
# power = Power(Config.POWER_ENABLE_PIN)

led = LED(Config.LED_COUNT)
tts = TTS()
led = LED(Config.get('neopixel','count'))
tts = TTS(translator=translator)

if Config.MOTION_PIN is not None:
motion = Sensor(Config.MOTION_PIN, pi=gpio)
if Config.get('motion','pin') != '':
motion = Sensor(Config.get('motion','pin'), pi=gpio)

pub.sendMessage('tts', msg='I am awake')

Expand All @@ -112,15 +117,15 @@ def main():
if len(sys.argv) > 1 and sys.argv[1] == 'preview':
preview = True

if Config.VISION_TECH is 'opencv':
if Config.get('vision','tech') is 'opencv':
camera_resolution = (640, 480) #(1024, 768) #- this halves the speed of image recognition
video_stream = VideoStream(resolution=camera_resolution).start()
vision = Vision(video_stream, mode=Vision.MODE_FACES, path=path, preview=preview, resolution=camera_resolution)
tracking = Tracking(vision)
training = TrainModel(dataset=path + '/matches/trained', output='encodings.pickle')
# timelapse = Timelapse(video_stream, path=path, original_resolution=camera_resolution)
elif Config.VISION_TECH is 'coral':
if Config.DEBUG_VISION:
elif Config.get('vision','tech') is 'coral':
if Config.get('vision', 'debug'):
# Testing - for fine-tuning tracking without the other stuff
pub.sendMessage('wake')
pub.sendMessage('power:use')
Expand All @@ -129,10 +134,10 @@ def main():
pub.sendMessage("servo:pan:mvabs", percentage=60)
sleep(1)

vision = Vision(preview=preview, mode=Config.VISION_MODE)
vision = Vision(preview=preview, mode=Config.get('vision','initial_mode'))
tracking = Tracking()

if Config.DEBUG_VISION:
if Config.get('vision', 'debug'):
while True:
pass

Expand All @@ -144,19 +149,19 @@ def main():
temp = PiTemperature()

# Voice
if Config.HOTWORD_MODEL is not None:
hotword = HotWord(Config.HOTWORD_MODEL)
if Config.get('hotword', 'model') is not '':
hotword = HotWord(Config.get('hotword', 'model'))
hotword.start() # @todo this starts the thread. can it be moved into hotword?
hotword.start_recog(sleep_time=Config.HOTWORD_SLEEP_TIME)
hotword.start_recog(sleep_time=Config.get('hotword', 'sleep_time'))
sleep(1) # @todo is this needed?
# @todo this is throwing errors: ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'defaults.bluealsa.device'

speech = SpeechInput()
# Output
if Config.BUZZER_PIN is not None:
speak = Braillespeak(Config.BUZZER_PIN, duration=80/1000)
if Config.get('buzzer', 'pin') != '':
speak = Braillespeak(Config.get('buzzer', 'pin'), duration=80/1000)

buzzer = Buzzer(Config.BUZZER_PIN)
buzzer = Buzzer(Config.get('buzzer', 'pin'))
animate = Animate()

# @todo 2k resistor needs switching to > 3k for 20v+ support.
Expand Down
Loading