by Emil² - Ξ
We are trying to build data gloves, that enable typing on a 34-key keyboard, without an actual keyboard, based on 6-axis IMU sensors, using machine learning.
LapCal/ ├─ Software/ │ ├─ data-collection.org │ ├─ *machine-learning.org* ├─ Documentation/ ├─ Utilities/ │ ├─ keyboard-visualization/ ├─ Hardware/ │ ├─ Firmware/ │ ├─ Electronics/
Because we are both Emacs users, we pivoted to writing our READMEs in org-mode, as well as using org-babel for literate programming, to replace jupyter notebooks. More specific documentation can be found in this README.
First, we tried to connect all 6 IMUs to the same main-processor, an ESP32. This delivered a mediocre read-out frequency of about 10Hz at best. Collecting data was never done properly using this approach due to trying to use timestamps to sync the sensor and some sort of keyboard state data.
To simplify the problem, and offload computation at the same time, we are now using the constraints of the 34-keyboard, almost forcing each finger to a specific set of keys, and are trying to develop a model that only takes the data of a single finger and maps it to its respective keys. This would then also enable the use of an upgraded version of the SenseUnit (small PCBs with the IMUs on them that are attached to the fingers) with a dedicated processor on them, like an RP2040, perhaps. The mainboard could then be built around a standard keyboard MCU, that is usually used in mechanical keyboards.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
