If you are developing the 'server' side of things and want to try out the code you can run this without any hardware by having a null serial terminal. To do this, in a terminal run:

python resources/demoSerial.py

then in another terminal run

python relayekeysd.py --noserial

NB: Only tested on MacOS but should work on any posix system. For Windows simply give a COM port that doesn't exist.

The server (RPC server or daemon as we sometimes refer to it) is the component that opens up a connection to the COM port and sends the correct AT command to the board. You can control it with some arguments

When you use our installer it installs this as a service. If you run the code without installing (or you turn the service off for some reason) you can run it as relaykeysd.py or relaykeysd.exe

--noserial

Run the daemon and dont try and connect to hardware. If you are on linux/MacOS you can fake a serial port following these tips. If you are on Windows just fix a COM port in the config file or use the --dev option - just choose a non-existent COM port

--dev

Force the daemon to use a COM port rather than auto detecting one.

e.g.

python relaykeysd.py --noserial --dev=COM7

For more info see here

--debug

Sets a more verbose debugging output on the console.

--pidfile=file

Give a pidfile for the daemon to crate - or link to one.

Default: pidfile

--logfile=logfile

File to use as a log file for the debugging messages.

Default: logfile

--config=configfile

File to use as a config file. For more info see here

Default: relaykeys.cfg

--ble_mode=True|False

Use the daemon in wireless (ble_mode) or wired mode.

Default: false

RelayKeys is a mixture of a Open Hardware board that communicates in Bluetooth LE and acts as a BLE HID device (i.e. a keyboard/mouse) which can emulate all keyboard keys and their modifiers as well as mouse movements. (4). This is controlled by a serial connection from a device - either wired (i.e. as a USB dongle) or wireless (i.e. as a BLE Serial device). We have developed RPC server (2) for a desktop computer which simplifies software connecting to the serial bus and allows us to create a simpler way of sending commands to the device than AT commands. This is done either directly via the RPC Server (Daemon) or via a Command Line Interface (CLI). **** As a demonstration we also have a small graphical interface application (RK-Desktop also referred to as the Relaykeys-QT application) (1).

If you make software and want to benefit from the features of RelayKeys you can either write commands to the RPC server (2), or via our CLI (1) or if you wish direct to the serial interface (6). This can be wired serial or wireless if you put the device into wireless mode (5). Even if you wish to ignore our hardware we do ask Assistive Technology developers to consider copying the command structure at either one of these levels to allow for open development.

The Arduino based board is currently a Adafruit nrf52840 express or Adafruit nrf52840 itsybitsy.

Anatomy of the files

• arduino/ - firmware for the arduino board

• docs/ - VuePress docs for http://acecentre.github.io/relayKeys/. Build Script is docs-deploy.sh

• resources/ - Helper files for development (e.g. demoSerial.py which pretends to be an arduino board connected, viewComPorts which outputs the connected com ports), AAC software pagesets and all-keys.txt which is a output of all the keys supported by RelayKeys. Also note the development files - relaykeys-pygame and relaykeys.py - which are stripped down versions of the whole project. Useful for debugging and testing things without the daemon server. Just move relaykeys.py into the root directory to get it to work.

• blehid.py - Library/Import file used by relaykeysd. Includes the function that writes the AT command to the serial port, init the serial port, and sends keyboard/mouse codes.

• build.py, buildinstaller.nsi - Build files to turn this project into a installer. Run python build.py to trigger the build.

• docs-deploy.sh - deploys the documentation to the github pages site.

• relaykeys-cli.py - This is the command line interface version of relaykeys. Conencts to the daemon server.

• relaykeys-example.cfg - The default config file. Note this gets copied on install to the right location and is used as the standard config. Note commented out baud and dev lines. Dev fixes the COM port if the com port finding code doesn't work

• relaykeys-qt.py - The GUI version of our relakeys testing app.

• relaykeys.spec.ini - Used for the NSIS build (i.e. buildinstaller.nsi)

• relaykeysclient.py - The main library file used by -cli and -qt versions. Connects to the server.

• relaykeysd-service-restart.bat, relaykeysd-service.py - The Windows service applcation

• relaykeysd.py - The daemon. This is what is run in the background - and controlled by the service. If you want to test RelayKeys - run this file first - and leave it running. e.g. python relakyeysd.py &&. Note the command line flags in the header.

Steps to run RelayKeys (non-installer method)

Prerequisites

• Install Python 3

• Have access to a nrf52840 arduino board. e.g. the Adafruit nrf52840 express

1. Grab a nrf52840 board and load the arduino code onto it. Plug it in a usb slot on your computer

2. Check your arduino is working as expected. Find and make a note of the COM port it is (found in the device manager)

3. Download the code to a sensible location - e.g. git clone https://github.com/AceCentre/RelayKeys.git

4. Copy the relaykeys-example.cfg file e.g. copy relaykeys-example.cfg relaykeys.cfg

5. Install requirements e.g. pip install -r requirements.txt

6. Run the Daemon code. python relaykeysd.py

7. Pair your relaykeys arduino with a PC/Mac/iOS/Android device and open a text file

8. Test it out with the cli file e.g. python relaykeys-cli.py type:Hello

You should, if everything is working as expected, see it type on the second device. If not have a look at the log files.

One problem may be that the daemon cant find the COM port. You can fix this by fixing the com port you found in step 2 and putting it in the config file. e.g. add dev=COM6 if the com port is 6. More information can be found here

Introduction

In the same directory as relaykeys there is a file relaykeys.cfg

It looks like this:

Feel free to change any of the settings but Be careful that you make the username/password the same - otherwise the CLI programme can't talk to the service.

Dev - Defining your port of the RelayKeys hardware

RelayKeys software tries to find the RelayKeys board automagically. If you have a device with a number of COM ports attached - or ones that have similar functionality you may have some difficulties. If so try fixing the COM port.

1. Access Device Manager. Search for "Device Manager" in Windows.

1. Click on Ports. If you only have RelayKeys connected you should see one device. If you see many - try unplugging it and replugging it in to find which COM port it is.

1. Note the number and edit your RelayKeys cfg file. Search for "RelayKeys config" in Windows - and open with Notepad.

1. Type dev = COM3 where COM3 is your port number you found in step 2-3.

Builds are focused on Windows - but we have started to work on a MacOS build.

For Windows

You need nsis installed and install SimpleSC Plugin: https://nsis.sourceforge.io/NSIS_Simple_Service_Plugin. Then

pip install -r requirements.txt
pip install -r equirements-build.txt 
python build.py 

You will then get a setup.exe

If you wish to create a UF2 file for the firmware we follow this guide (see "Making your own UF2") - making note that we are on M4 based boards.

Our small team truly appreciates every contribution made by our community: user stories, feature requests, bug reports, and especially pull requests!. If you have any questions please reach out to our Core team at AceCentre.

User stories

So you use RelayKeys? Like what we are doing? Got a real problem that needs fixing but don't understand all this code stuff? Please get in touch. We will try and help - but please note; this is very much a open source, and fundraised project. If you can please consider donating the project

RelayKeys Repository

acecentre/relaykeys

This is the home to the project. Please fork and make changes on this project going forward. A quick overview of the contents:

• arduino/ contains sketches to make the board work. A massive thanks to Adafruit as the nrf52840 is their board with their firmware and this code is largley their example code. We have added mouse functionality

• docs/ The folder containing the docs (these docs that you are reading!). Its all made with gitbook

• resources/ a dumping ground of resources/tools that may be useful in development. Of Note is demoSerial.py - a way of programming this without the board to hand on Linux/Mac. Also see viewComPorts.py to debug your COM Ports

• blehid.py - this is the module used in relaykeysclient and relaykeysd. If you want to view things like keycode conversion and the like - see here.

• buildinstaller.py - this is a script that "builds" the pyinstaller binaries - and the NSIS setup.exe.

• relaykeys-cli.py - the CLI programme

• relaykeys.py - the original code we used to test this. Redundant largely now - but if you want to get your head around how all this works look here first

Simple Pull Requests

Before we get into the full-blown "proper" way to do a pull request, let's quickly cover an easier method you can use for small fixes. This way is especailly useful for fixing quick typos in the docs, but is not as safe for code changes since it bypasses validation and linting.

1. Sign in to GitHub

2. Go to the file you want to edit (eg: this page)

3. Click the pencil icon to "Edit this file"

4. Make any changes

5. Describe and submit your changes within "Propose file change"

That's it! GitHub will create a fork of the project for you and submit the change to a new branch in that fork. Just remember to submit separate pull requests when solving different problems.

Proper Pull Requests

Loosely based on this great Gist by Chaser324

We like to keep a tight flow when working with GitHub to make sure we have a clear history and accountability of what changes were made and when. Working with Git, and especially the GitHub specific features like forking and creating pull requests, can be quite daunting for new users.

To help you out in your Git(Hub) adventures, we've put together the (fairly standard) flow of contributing to an open source repo.

Forking the repo

Whether you're working on the API or the App, you will need to have your own copy of the codebase to work on. Head to the repo of the project you want to help out with and hit the Fork button. This will create a full copy of the whole project for you on your own account.

To work on this copy, you can install the project locally according to the normal installation instructions, substituting the name acecentre with the name of your github account.

Keeping your fork up to date

If you're doing more work than just a tiny fix, it's a good idea to keep your fork up to date with the "live" or upstream repo. This is the main acecentre repo that contains the latest code. If you don't keep your fork up to date with the upstream one, you'll run into conflicts pretty fast. These conflicts will arise when you made a change in a file that changed in the upstream repo in the meantime.

On git remotes

When using git on the command line, you often pull and push to origin. You might have seen this term in certain commands, like

git push origin master

or

git pull origin new-feature

In this case, the word origin is refered to as a remote. It's basically nothing more than a name for the full git url you cloned the project from:

git push origin master

is equal to

git push git@github.com:username/repo.git master

A local git repo can have multiple remotes. While it's not very common to push your code to multiple repo's, it's very useful when working on open source projects. It allows you to add the upstream repo as another remote, making it possible to fetch the latest changes straight into your local project.

# Add 'upstream' to remotes
git remote add upstream git@github.com:acecentre/relaykeys.git

When you want to update your fork with the latest changes from the upstream project, you first have to fetch all the (new) branches and commits by running

git fetch upstream

When all the changes are fetched, you can checkout the branch you want to update and merge in the changes.

git checkout master
git rebase upstream/master

If you haven't made any commits on the branch you're updating, git will update your branch without complaints. If you have created commits in the meantime, git will step by step apply all the commits from upstream and try to add in the commit you made in the meantime. It is very plausible that conflicts arise at this stage. When you've changed something that also changed on the upstream, git requires you to resolve the conflict yourself before being able to move on.

::: danger Conflicts You should always favor changes on upstream over your local ones. :::

Doing Work

Whenever you begin working on a bugfix or new feature, make sure to create a new branch. This makes sure that your changes are organized and separated from the master branch, so you can submit and manage your pull requests for separate fixes/features more easily.

# Checkout the master branch - you want your new branch to come from master
git checkout master

# Create a new branch named newfeature (give your branch its own simple informative name)
git branch newfeature

# Switch to your new branch
git checkout newfeature

::: warning Up-to-date Make sure to update your master branch with the one from upstream, so you're certain you start with the latest version of the project! :::

Submitting a Pull Request

Prior to opening your pull request, you might want to update your branch a final time, so it can immediately be merged into the master branch of upstream.

# Fetch upstream master and merge with your repo's master branch
git fetch upstream
git checkout master
git merge upstream/master

# If there were any new commits, rebase your master branch
git checkout newfeature
git rebase master

::: warning Make sure to check if your branch is up to date with the master branch of upstream. An outdated branch makes it near impossible for the maintainers of acecentre to check and review the pull request and will most likely result in a delayed merge. :::

Once you've commited and pushed all the changes on your branch to your fork on GitHub, head over to GitHub, select your branch and hit the pull request button.

You can still push new commits to a pull request that already has been opened. This way, you can fix certain comments reviewers might have left.

::: tip Please allow the maintainers of upstream to push commits to your fork by leaving the "Allow edits from maintainers" option turned on. This allows our Core Team to help out in your PR! :::

Feature Requests

80/20 Rule

The main thing to be aware of when submitting a new acecentre feature request, is our rule on edge-cases. To keep the acecentre core codebase as clean and simple as possible we will only consider adding features that at least 80% of our user-base will use. If we feel that less than 80% of our users will find the feature valuable then we will not implement it. Instead, those edge-case features should be added as Extensions.

Browsing Existing Requests

Before adding a new request, you should also first search to see if it has already been submitted. All feature requests should include the enhancement label, so you can filter by that. And remember to also check closed issues since your feature might have already been submitted in the past and either rejected or already implemented.

Also, if you want to see the most highly requested features you can sort by :+1: (the thumbs-up emoji).

Submitting a Request

If your idea passes the 80/20 test and has not already been submitted, then we'd love to hear it! Submit a new issue using the Feature Request template and be sure to include the enhancement label. It's important to completely fill our the template with as much useful information as possible so that we can properly review your request. If you have screenshots, designs, code samples, or any other helpful assets be sure to include those too!

Voting on Requests

You can also vote on existing feature requests. As mentioned above, the :+1: and :-1: are used for sorting, so adding one of these reactions to the GitHub issue will cast a vote that helps us better identify the most desired (or undesired) features. And remember to add a comment if you have additional thoughts to help clarify or improve the request.

Fulfilling a Request

Our core team is always working hard to implement the most highly-requested community features, but we're a small team. If you need the feature faster than we can provide it, or simply want to help improve the acecentre platform, we'd love to receive a pull-request from you!

• Adafruit - for being awesome. They have made a super duper board - with love AT Commands

• bbx10 on the Adafruit forums who got this up and running. Awesome.

• Keyboard by Atif Arshad from the Noun Project

• Bluetooth by Adrien Coquet from the Noun Project

Other projects / Similar work / Inspiration

• The original RelayKeys from Harold Pimental.

• bbx10 on the adafruit forums. bbx10 developed the Ascii to HID translation function. A massive thanks - the code is currently mostly his. He also worked out some of the early problems on speed issues we were having. You can read the full thread here.

• HID-Relay from juancgarcia. Not really spent much time looking at this - but looks neat. Converts hardware keyboards to Bluetooth.

• 232Key - Converts Serial devices to Keyboard. Kind of the other way round to what we want.

• BL_keyboard_RPI. Turns a Pi into keyboard emulator

• ESP32_mouse_keyboard. Uses a ESP32 as a mouse/keyboard over serial. Very similar idea. (See issue 39for details about this in relation to using VNC (TY @RoganDawes)

AAC projects

• MacroServerMac was an attempt to create a Mac Port of "MacroServer" developed by JabblaSoft for MindExpress . This is a protocol for communication over a TCP/IP stack. Its pretty nice - but if you are in a school or business allowing others machines to access the network in this way is often restricted. It can also be pretty flaky

• Liberator / PRC - have the neatest commercial solution out there for AAC. You can either plug in a USB cable - or use a bluetooth dongle to connect with another computer. Its awesome - but sadly only available to you if use one of their devices.

• Dynavox used to make the AccessIT. A similar idea but using infrared rather than radio/bluetooth. It was pretty expensive but a lot of people loved its simplicity. More recently they have brought this back to life with AccessIT 3. Note it only works in Snap software - and not for any device. Just another windows device. You also need a USB port on the device you connect to.

• The AAC world has been trying to create standards for this for years.. and some have succeeded. Check out AACKeys and the "GIDEI" protocol - which now feels a little outdated but a great attempt at standardising communication between AAC devices and other systems over serial.

Non-disability related products

• the Buffalo BSHSBT04BK was pretty neat. You can still get this in Japan and does a very similar job

• The IOGEAR KeyShair (now discontinued) looked like exactly the same dongle - but with different software.

Both of these products though failed to respond to software (on-screen) keyboards reliably.