Volume 9, Issue 5 (September & October 2018 2018)
BCN 2018, 9(5): 373-388 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Sadeghi S, Maleki A. Recent Advances in Hybrid Brain-Computer Interface Systems: A Technological and Quantitative Review. BCN 2018; 9 (5) :373-388
URL: http://bcn.iums.ac.ir/article-1-960-en.html
URL: http://bcn.iums.ac.ir/article-1-960-en.html
1- Department of Biomedical Engineering, Faculty of New Sciences and Technologies, Semnan University, Semnan, Iran.
Abstract:
Brain-Computer Interface (BCI) is a system that enables users to transmit commands to the computer using their brain activity recorded by electroencephalography. In a Hybrid Brain-Computer Interface (HBCI), a BCI control signal combines with one or more BCI control signals or with Human-Machine Interface (HMI) biosignals to increase classification accuracy, boost system speed, and improve user’s satisfaction. HBCI systems are categorized according to the type of combined signals and the combination technique (simultaneous or sequential). They have been used in several applications such as cursor control, target selection, and spellers. Increasing the number of articles published in this field indicates the significance of these systems. In this paper, different HBCI combinations, their important features, and potential applications are discussed. In most cases, the combination of a BCI control signal with a HMI biosignal yields higher information transfer rate than two BCI control signals.
Keywords: Brain-Computer Interfaces (BCI), BCI control signal, Human-machine interface biosignal, Simultaneous and sequential HBCI
Type of Study: Methodological Notes |
Subject:
Computational Neuroscience
Received: 2017/06/6 | Accepted: 2018/05/29 | Published: 2018/09/1
Received: 2017/06/6 | Accepted: 2018/05/29 | Published: 2018/09/1
References
1. Ahn, S., Ahn, M., Cho, H., & Jun, S. C. (2013). hybrid brain-computer interface based on motor imagery and tactile selective attention. In N.L. Millán, R. Leeb, J.D.R. Millán, N. Guechoul, T. Carlson, A. Sobolewski, et al. (Eds.), Proceedings of TOBI Workshop IV (pp. 51-52). Switzerland: École polytechnique fédérale de Lausanne. [PMCID]
2. Faller, J. (2012). BCIs that use steady-state visual evoked potentials or slow cortical potentials. In J. Wolpaw, & E. Winter Wolpaw (Eds.), Brain-Computer Interfaces, Principles and Practice. Oxford: Oxford University Press.
3. Allison, B. Z., Brunner, C., Altstätter, C., Wagner, I. C., Grissmann, S., & Neuper, C. (2012). A hybrid ERD/SSVEP BCI for continuous simultaneous two dimensional cursor control. Journal of Neuroscience Methods, 209(2), 299-307. [DOI:10.1016/j.jneumeth.2012.06.022] [PMID] [DOI:10.1016/j.jneumeth.2012.06.022]
4. Allison, B. Z., Brunner, C., Kaiser, V., Müller-Putz, G. R., Neuper, C., & Pfurtscheller, G. (2010). Toward a hybrid brain-computer interface based on imagined movement and visual attention. Journal of Neural Engineering, 7(2), 026007. [DOI:10.1088/1741-2560/7/2/026007] [PMID] [DOI:10.1088/1741-2560/7/2/026007]
5. Al Shargie, F., Kiguchi, M., Badruddin, N., Dass, S. C., Hani, A. F. M., & Tang, T. B. (2016). Mental stress assessment using simultaneous measurement of EEG and fNIRS. Biomedical Optics Express, 7(10), 3882-98. [DOI:10.1364/BOE.7.003882] [PMID] [PMCID] [DOI:10.1364/BOE.7.003882]
6. Amiri, S., Rabbi, A., Azinfar, L., & Fazel-Rezai, R. (2013). A review of P300, SSVEP, and hybrid P300/SSVEP brain-computer interface systems. In R. Fazel-Rezai (Eds.), Brain-Computer Interface Systems-Recent Progress and Future Prospects. Intech Open. [DOI:10.5772/56135] [DOI:10.5772/56135]
7. Bai, O., Lin, P., Huang, D., Fei, D. Y., & Floeter, M. K. (2010). Towards a user-friendly brain-computer interface: Initial tests in ALS and PLS patients. Clinical Neurophysiology, 121(8), 1293-1303. [DOI:10.1016/j.clinph.2010.02.157] [PMID] [PMCID] [DOI:10.1016/j.clinph.2010.02.157]
8. Bharne, P. P., & Kapgate, D. (2015). Hybrid visual BCI combining SSVEP and P300 with high ITR and accuracy. International Journal of Computer Science and Mobile Computing, 4(6), 1-5.
9. Bhattacharyya, S., Konar, A., & Tibarewala, D. N. (2014). "Motor imagery, P300 and error-related EEG-based robot arm movement control for rehabilitation purpose. Medical & Biological Engineering & Computing, 52(12): 1007-17. [DOI:10.1007/s11517-014-1204-4] [PMID] [DOI:10.1007/s11517-014-1204-4]
10. Bi, L., Lian, J., Jie, K., Lai, R., & Liu, Y. (2014). A speed and direction-based cursor control system with P300 and SSVEP. Biomedical Signal Processing and Control, 14, 126-33. [DOI:10.1016/j.bspc.2014.07.009] [DOI:10.1016/j.bspc.2014.07.009]
11. Breitwieser, C., Pokorny, C., & Gernot, R. M. (2016). A hybrid three-class brain? Computer interface system utilizing SSSEPs and transient ERPs. Journal of Neural Engineering, 13(6), 066015. [DOI:10.1088/1741-2560/13/6/066015] [PMID] [DOI:10.1088/1741-2560/13/6/066015]
12. Brunner, C., Allison, B. Z., Altstätter, C., & Neuper, C. (2011). A comparison of three brain–computer interfaces based on event-related desynchronization, steady state visual evoked potentials, or a hybrid approach using both signals. Journal of Neural Engineering, 8(2), 025010. [DOI:10.1088/1741-2560/8/2/025010] [PMID] [DOI:10.1088/1741-2560/8/2/025010]
13. Buccino, A. P., Keles, H. O., & Omurtag, A. (2016). Hybrid EEG-fNIRS asynchronous brain-computer interface for multiple motor tasks. PloS One, 11(1), e0146610. [DOI:10.1371/journal.pone.0146610] [PMID] [PMCID] [DOI:10.1371/journal.pone.0146610]
14. Cao, L., Li, J., Ji, H., & Jiang, C. (2014). A hybrid brain computer interface system based on the neurophysiological protocol and brain-actuated switch for wheelchair control. Journal of Neuroscience Methods, 229, 33-43. [DOI:10.1016/j.jneumeth.2014.03.011] [PMID] [DOI:10.1016/j.jneumeth.2014.03.011]
15. Capati, F. A., Bechelli, R. P., & Castro, M. C. F. (2016). Hybrid SSVEP/P300 BCI keyboard. In J. Gilbert, H. Azhari, H. Ali, C. Quintão, J. Sliwa, C. Ruiz, et al., (Eds.), Proceedings of the International Joint Conference on Biomedical Engineering Systems and Technologies (pp. 214-8). Setúbal: SciTePress-Science and Technology Publications. [PMID] [PMID]
16. Chang, M. H., Lee, J. S., Heo, J., & Park, K. S. (2016). Eliciting dual-frequency SSVEP using a hybrid SSVEP-P300 BCI. Journal of Neuroscience Methods, 258, 104-13. [DOI:10.1016/j.jneumeth.2015.11.001] [PMID] [DOI:10.1016/j.jneumeth.2015.11.001]
17. Coyle, S., Ward, T., Markham, C., & McDarby, G. (2004). On the Suitability of Near-Infrared (NIR) systems for next-generation brain-computer interfaces. Physiological Measurement, 25(4), 815. [DOI:10.1088/0967-3334/25/4/003] [PMID] [DOI:10.1088/0967-3334/25/4/003]
18. Dong, X., Wang, H., Chen, Z., & Shi, B. E. (2015). Hybrid brain computer interface via Bayesian integration of EEG and eye gaze. Paper presented at the 17th International IEEE/EMBS Conference on Neural Engineering (NER), Montpellier, France, 2 July 2015. [DOI:10.1109/NER.2015.7146582] [DOI:10.1109/NER.2015.7146582]
19. Edlinger, G., Holzner, C., & Guger, C. (2011). A hybrid brain-computer interface for smart home control. Paper presented at the International Conference on Human-Computer Interaction, Las Vegas, 15-20 July 2018. [DOI:10.1007/978-3-642-21605-3_46] [DOI:10.1007/978-3-642-21605-3_46]
20. Évain, A., Argelaguet, F., Casiez, G., Roussel, N., & Lécuyer, A. (2016). Design and evaluation of fusion approach for combining brain and gaze inputs for target selection. Frontiers in Neuroscience, 10, 454. [DOI:10.3389/fnins.2016.00454] [PMID] [PMCID] [DOI:10.3389/fnins.2016.00454]
21. Fazel-Rezai, R., & Ahmad, W. (2011). P300-based brain-computer interface paradigm design. In U. Hoffmann, J.M. Vesin, T. Ebrahimi (Eds.), Recent Advances in Brain-Computer Interface Systems. IEEE Xplore Digital Library.
https://doi.org/10.5772/579 [DOI:10.5772/14858]
22. Fazel Rezai, R., Allison, B. Z., Guger, C., Sellers, E. W., Kleih, S. C., & Kübler, A. (2012). P300 brain computer interface: Current challenges and emerging trends. Frontiers in Neuroengineering, 5, 14. [DOI:10.3389/fneng.2012.00014] [PMID] [PMCID] [DOI:10.3389/fneng.2012.00014]
23. Fazli, S., Mehnert, J., Steinbrink, J., Curio, G., Villringer, A., Müller, K. R., et al. (2012). Enhanced performance by a hybrid NIRS–EEG brain computer interface. Neuroimage, 59(1), 519-29. [DOI:10.1016/j.neuroimage.2011.07.084] [PMID] [DOI:10.1016/j.neuroimage.2011.07.084]
24. Finke, A., Knoblauch, A., Koesling, H., & Ritter, H. (2011). A hybrid brain interface for a humanoid robot assistant. Paper presented at 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, 30 August-3 September, 2011. [DOI:10.1109/IEMBS.2011.6091728] [PMID] [DOI:10.1109/IEMBS.2011.6091728]
25. Guger, C., Allison, B. Z., Großwindhager, B., Prückl, R., Hintermüller, C., Kapeller, C., et al. (2012). How many people could use an SSVEP BCI?. Frontiers in Neuroscience, 6, 169. [DOI:10.3389/fnins.2012.00169] [PMID] [PMCID] [DOI:10.3389/fnins.2012.00169]
26. Guger, C., Allison, B. Z., & Müller Putz, G. R. (2015). Brain-Computer Interface Research: A state-of-the-art summary 4. In Ch. Guger, B.Z. Allison, G. Edlinger (Eds.), Brain-Computer Interface Research (pp. 1-8). Berlin: Springer. [DOI:10.1007/978-3-319-25190-5] [DOI:10.1007/978-3-319-25190-5]
27. Herff, C., Fortmann, O., Tse, C. Y., Cheng, X., Putze, F., Heger, D., & Schultz, T. (2015). Hybrid fNIRS-EEG based discrimination of 5 levels of memory load. Paper presented at 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER). Montpellier, France, 2-24 April 2015. [DOI:10.1109/NER.2015.7146546] [DOI:10.1109/NER.2015.7146546]
28. Ji, H., Li, J., Lu, R., Gu, R., Cao, L., & Gong, X. (2016). EEG classification for hybrid brain-computer interface using a tensor based multiclass multimodal analysis scheme. Computational intelligence and neuroscience, 2016, 51. [DOI:10.1155/2016/1732836] [PMID] [PMCID] [DOI:10.1155/2016/1732836]
29. Khan, M. J., Hong, K. S., Naseer, N., Bhutta, M. R., & Yoon, S. H. (2014a). Hybrid EEG-NIRS BCI for rehabilitation using different-source brain signals. Paper presented at the Annual Conference of the Society of Instrument and Control Engineers (SICE), Sapporo, Japan, 9 September 2014.
30. Khan, M. J., Hong, M. J., & Hong, K. S. (2014b). Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface. Frontiers in Human Neuroscience, 8, 244. [DOI:10.3389/fnhum.2014.00244] [PMID] [PMCID] [DOI:10.3389/fnhum.2014.00244]
31. Kiguchi, K., & Hayashi, Y. (2012). A study of EMG and EEG during perception-assist with an upper-limb power-assist robot. Paper presented at Robotics and Automation (ICRA), Saint Paul, 14-18 May 2012. [DOI:10.1109/ICRA.2012.6225027] [DOI:10.1109/ICRA.2012.6225027]
32. Kim, M., Kim, B. H., & Jo, S. (2015). Quantitative evaluation of a low-cost noninvasive hybrid interface based on EEG and eye movement. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 23(2), 159-68. [DOI:10.1109/TNSRE.2014.2365834] [PMID] [DOI:10.1109/TNSRE.2014.2365834]
33. Kim, Y., & Jo, S. (2015). Wearable hybrid brain-computer interface for daily life application. Paper presented at 2015 3rd International Winter Conference on Brain-Computer Interface, Sabuk, South Korea, 12-14 January 2015. [DOI:10.1109/IWW-BCI.2015.7073029]
34. Koo, B., Lee, H. G., Nam, Y., Kang, H., Koh, C. S., Shin, H. C., et al. (2015a). A hybrid EOG-P300 BCI with dual monitors. Paper presented at 2014 International Winter Workshop on Brain-Computer Interface (BCI), Jeongsun-kun, South Korea, 17-19 February 2014.
35. Koo, B., Lee, H. G., Nam, Y., Kang, H., Koh, C. S., Shin, H. C., et al. (2015b). A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery. Journal of Neuroscience Methods, 244, 26-32. [DOI:10.1016/j.jneumeth.2014.04.016] [PMID] [DOI:10.1016/j.jneumeth.2014.04.016]
36. Lalitharatne, T. D., Teramoto, K., Hayashi, Y., & Kiguchi, K. (2013). Towards hybrid EEG-EMG-based control approaches to be used in bio-robotics applications: Current status, challenges and future directions. Paladyn, Journal of Behavioral Robotics, 4(2), 147-54. [DOI:10.2478/pjbr-2013-0009] [DOI:10.2478/pjbr-2013-0009]
37. Lee, M. H., Fazli, S., Mehnert, J., & Lee, S. W. (2014). Hybrid brain-computer interface based on EEG and NIRS modalities. Paper presented at 2014 International Winter Workshop on Brain-Computer Interface (BCI), Jeongsun-kun, South Korea, 17-19 February 2014. [DOI:10.1109/iww-BCI.2014.6782577] [DOI:10.1109/iww-BCI.2014.6782577]
38. Leeb, R., Sagha, H., & Chavarriaga, R. (2010). Multimodal fusion of muscle and brain signals for a hybrid-BCI. Peper presented at 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, Buenos Aires, Argentina, 31 August- 4 September 2010. [DOI:10.1109/IEMBS.2010.5626233] [PMID] [DOI:10.1109/IEMBS.2010.5626233]
39. Leeb, R., Sagha, H., Chavarriaga, R., & del R Millán, J. (2011). A hybrid brain-computer interface based on the fusion of electroencephalographic and electromyographic activities. Journal of Neural Engineering, 8(2), 025011. [DOI:10.1088/1741-2560/8/2/025011] [PMID] [DOI:10.1088/1741-2560/8/2/025011]
40. Li, J., Ji, H., Cao, L., Gu, R., Xia, B., & Huang, Y. (2013). Wheelchair control based on multimodal brain-computer interfaces. Peper presented at International Conference on Neural Information Processing, Daegu, Korea, 3-7 November 2013. [DOI:10.1007/978-3-642-42054-2_54] [DOI:10.1007/978-3-642-42054-2_54]
41. Li, J., Ji, H., Cao, L., Zang, D., Gu, R., Xia, B., et al. (2014). Evaluation and application of a hybrid brain computer interface for real wheelchair parallel control with multi-degree of freedom. International Journal of Neural Systems, 24(04), 1450014. [DOI:10.1142/S0129065714500142] [PMID] [DOI:10.1142/S0129065714500142]
42. Lin, K., Chen, X., Huang, X., Ding, Q., & Gao, X. (2015). A hybrid BCI speller based on the combination of EMG envelopes and SSVEP. Heidelberg: Springer Berlin Heidelberg. [DOI:10.1186/s40535-014-0004-0] [DOI:10.1186/s40535-014-0004-0]
43. Lin, K., Cinetto, A., Wang, Y., Chen, X., Gao, S., & Gao, X. (2016). An online hybrid BCI system based on SSVEP and EMG. Journal of Neural Engineering, 13(2), 026020. [DOI:10.1088/1741-2560/13/2/026020] [PMID] [DOI:10.1088/1741-2560/13/2/026020]
44. Liu, Y. H., Wang, S. H., & Hu, M. R. (2016). A self-paced P300 healthcare brain-computer interface system with SSVEP-based switching control and kernel FDA+SVM-based detector. Applied Sciences, 6(5), 142. [DOI:10.3390/app6050142] [DOI:10.3390/app6050142]
45. Liu, H., Li, Y., Liu, H., & Wang, S. (2016). Bagging regularized common spatial pattern with hybrid motor imagery and myoelectric signal. Paper presented at 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, 20-25 March 2016. [DOI:10.1109/ICASSP.2016.7471793] [DOI:10.1109/ICASSP.2016.7471793]
46. Long, J., Li, Y., Wang, H., Yu, T., Pan, J., & Li, F. (2012a). A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 20(5), 720-9. [DOI:10.1109/TNSRE.2012.2197221] [PMID] [DOI:10.1109/TNSRE.2012.2197221]
47. Long, J., Li, Y., Yu, T., & Gu, Z. (2012b). Target selection with hybrid feature for BCI-based 2-D cursor control. IEEE Transactions on Biomedical Engineering, 59(1), 132-40. [DOI:10.1109/TBME.2011.2167718] [PMID] [DOI:10.1109/TBME.2011.2167718]
48. Ma, J., Zhang, Y., Cichocki, A., & Matsuno, F. (2015). A novel EOG/EEG hybrid human-machine interface adopting eye movements and ERPs: Application to robot control. IEEE Transactions on Biomedical Engineering, 62(3), 876-89. [DOI:10.1109/TBME.2014.2369483] [PMID] [DOI:10.1109/TBME.2014.2369483]
49. Ma, L., Zhang, L., Wang, L., Xu, M., Qi, H., Wan, B., et al. (2012). A hybrid brain-computer interface combining the EEG and NIRS. Peper presented at IEEE International Conference on Virtual Environments Human-Computer Interfaces and Measurement Systems (VECIMS) Proceedings, Tianjin, China, 2-4 July 2012. [DOI:10.1109/VECIMS.2012.6273214] [DOI:10.1109/VECIMS.2012.6273214]
50. Meena, Y., Prasad, G., Cecotti, H., & Wong-Lin, K. (2015). Simultaneous gaze and motor imagery hybrid BCI increases single-trial detection performance: A compatible incompatible study. Paper presented at 9th IEEE-EMBS International Summer School on Biomedical Signal Processing, Pavia, Italy, September 2015.
51. Mercado, L., Rodríguez-Li-án, A., Torres-Trevi-o, L. M., & Quiroz, G. (2016). Hybrid BCI approach to control an artificial tibio-femoral joint. Paper presented at 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, 16-20 August 2016. [DOI: 10.1109/EMBC.2016.7591302] [DOI:10.1109/EMBC.2016.7591302]
52. Müller-Putz, G. R., Leeb, R., Millán, J. D. R., Horki, P., Kreilinger, A., Bauernfeind, et al. (2012). Principles of hybrid brain-computer interfaces. In B.Z. Allison, S. Dunne, R. Leeb, J. D. R. Millán, A. Nijholt (Eds.), Towards Practical Brain-Computer Interfaces (pp. 355-373). Berlin: Springer.
53. [DOI:10.1007/978-3-642-29746-5_18] [DOI:10.1007/978-3-642-29746-5_18]
54. Panicker, R. C., Puthusserypady, S., & Sun, Y. (2011). An asynchronous P300 BCI with SSVEP-based control state detection. IEEE Transactions on Biomedical Engineering 58(6), 1781-8. [DOI:10.1109/TBME.2011.2116018] [PMID] [DOI:10.1109/TBME.2011.2116018]
55. Peng, N., Zhang, R., Zeng, H., Wang, F., Li, K., Li, Y., et al. (2016). Control of a nursing bed based on a hybrid brain-computer interface. Paper presented at 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, 16-20 August 2016. [DOI:10.1109/EMBC.2016.7591008] [DOI:10.1109/EMBC.2016.7591008]
56. Pfurtscheller, G., & Da Silva, F. L. (1999). Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical Neurophysiology, 110(11), 1842-57. [DOI:10.1016/S1388-2457(99)00141-8] [DOI:10.1016/S1388-2457(99)00141-8]
57. Pfurtscheller, G., Solis-Escalante, T., Ortner, R., Linortner, P., & Muller-Putz, G. R. (2010). Self-paced operation of an SSVEP-Based orthosis with and without an imagery-based "brain switch:" A feasibility study towards a hybrid BCI. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 18(4), 409-14. [DOI:10.1109/TNSRE.2010.2040837] [DOI:10.1109/TNSRE.2010.2040837]
58. Pokorny, C., Breitwieser, C., & Müller-Putz, G. R. (2016). The role of transient target stimuli in a steady-state somatosensory evoked potential-based brain–computer interface setup. Frontiers in Neuroscienc
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
Copyright © The Author(s);
This is an open access article distributed under the terms of the Creative Commons Attribution License (CC-By-NC), which permits use, distribution, and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.
Contact Information
