Subproject D02

Subproject D02

Empatho-Kinaesthetic Sensor Technology for Biofeedback in Depressed Patients

The D02 project focuses on utilizing facial expressions as a biomarker in smartphone-based reappraisal training for psychological assessment. The project aims to understand the relationship between cognition, facial expressions, and affect as underlying mechanisms for the development and maintenance of depression. It highlights the lack of empirical studies on facial expressions and the need for quantifying and modifying them. The project presents a feasibility study on smartphone-based biofeedback Training for reducing depressive mood and suggests that this training could be a useful tool for individuals with depressive symptoms. It also proposes the establishment of empathokinaesthetic sensor technology and machine learning methods for the automatic detection and modification of Depression-associated facial expressions, posture, and movement. The mutual activation of depressogenic evaluation processes and depression-associated kinaesthesia is a significant factor in psychological information processing theories. It emphasizes the importance of understanding these underlying mechanisms in developing effective interventions for depression. Overall, the project provides insights into the potential of smartphone-based biofeedback training and machine-learning algorithms in detecting and treating depression.

 

                                                                                                            

Contacts

Prof. Dr. Matthias Berking

Principal Investigator

Prof. Dr. Björn Eskofier

Principal Investigator

 

Dr. Marie Keinert, M. Sc.

PostDoc

Lydia Rupp

Doctoral Candidate

Misha Sadeghi, M. Sc.

Doctoral Candidate

Amirreza Asemanrafat, M. Sc.

Associated Doctoral Candidate

Sharon Ernst, M.Sc.

Associated Doctoral Candidate

Malin Hager, M.Sc.

Associate Doctoral Candidate

Farnaz Rahimi, M. Sc.

Associate Doctoral Candidate

 

Prof. Dr. Bernhard Egger

Associated Member

Dr.-Ing. Robert Richer

PostDoc

Dr. Lena Schindler-Gmelch

Postdoc

 

 

Additional Information

  • Abel L., Richer R., Burkhardt F., Kurz M., Ringgold V., Schindler-Gmelch L., Eskofier B., Rohleder N.:
    Body movements as biomarkers: Machine Learning-based prediction of HPA axis reactivity to stress.
    Psychoneuroendocrinology 179 (2025), Art.Nr.: 107528. ISSN: 0306-4530. DOI: 10.1016/j.psyneuen.2025.107528.
  • Balbach S., Dorn C., Schindler-Gmelch L., Berking M., Hagelauer A.:
    Low-Power Wireless Multi-Channel Surface Electromyography Sensor for Facial Muscle Sensing.
    IEEE Wireless and Microwafe Technology Conference (WAMICON), Cocoa Beach, Florida, 2025.
  • Hager M., Keinert EM., Berking M.:
    STARK mit Depression: Ein multimodales Approach-Avoidance Modification Training für Jugendliche mit Depression – Ein Studienprotokoll.
    4. Deutscher Psychotherapie Kongress (Berlin, 2025).
  • Keinert EM., Schindler-Gmelch L., Rupp L., Sadeghi M., Richer R., Capito K., Eskofier B., Berking M.:
    The EmpkinS-EKSpression Reappraisal Training Augmented With Kinesthesia in Depression.
    JMIR Formative Research 9 (2025), Art.Nr.: e65357. ISSN: 2561-326X. DOI: 10.2196/65357.
  • Keinert EM., Berking M.:
    Declaring „war“ on stress: Preliminary results of a randomized controlled study investigating a speech-based approach-avoidance modification training targeting dysfunctional beliefs in individuals with elevated stress.
    Vortrag auf dem 4. Deutschen Psychotherapie Kongress (Berlin, 2025).
  • Keinert M., Pistrosch S., Mallol-Ragolta A., Schuller BW., Berking M.:
    Facial Emotion Recognition of 16 Distinct Emotions From Smartphone Videos: Comparative Study of Machine Learning and Human Performance.
    Journal of Medical Internet Research 27 (2025), Art.Nr.: e68942. ISSN: 1438-8871. DOI: 10.2196/68942.
  • Rupp L., Kumar A., Sadeghi M., Schindler-Gmelch L., Keinert M., Eskofier B., Berking M.:
    Stress can be detected during emotion-evoking smartphone use: a pilot study using machine learning.
    Frontiers in Digital Health 7 (2025), Art.Nr.: 1578917. ISSN: 2673-253X. DOI: 10.3389/fdgth.2025.1578917.
  • Rupp L., Schindler-Gmelch L., Sadeghi M., Rahimi F., Keinert EM., Hager M., Capito K., Richer R., Eskofier B., Berking M.:
    EmpkinS-EKSpression: Ergebnisse einer Randomisiert-kontrollierten klinischen Studie eines Emotions-gestützten Trainings zur Kognitiven Umstrukturierung.
    Vortrag auf dem 4. Deutschen Psychotherapie Kongress (Berlin, 2025).
  • Schindler-Gmelch L., Rupp L., Rahimi F., Sadeghi M., Keinert EM., Hager M., Capito K., Richer R., Eskofier B., Berking M.:
    EmpkinS-EKSpression-BioFACEBack: Ergebnisse eines randomisiert-kontrollierten Trials zu gesichtsmuskelfokussiertem Biofeedback bei depressiven Erkrankungen.
    Vortrag auf dem 4. Deutschen Psychotherapie Kongress (Berlin, 2025).
  • Ernst S., Krauss D., Moß M., Vossiek M., Eskofier B., Merking M.:
    Affect during sleep: Exploring the capabilities of Artificial Intelligence in Sleep Research.
    Proceedings of the International Medicine and Health Sciences Congress, 2024. DOI: 10.53375/imhsc.2024.117.
  • Fleischmann S., Holzwarth Correa V., Coppers B., Sadeghi M., Richer R., Kleyer A., Simon D., Bräunig J., Vossiek M., Schönau V., Schett G., Koelewijn A., Leyendecker S., Eskofier B., Liphardt AM.:
    Classification of rheumatoid arthritis from hand motion capture data using machine learning.
    13. Kongress der Deutschen Gesellschaft für Biomechanik (DGfB) (Heidelberg, 24 April 2024 – 26 April 2024).
  • Keinert EM., Schindler-Gmelch L., Rupp L., Sadeghi M., Capito K., Hager M., Rahimi F., Richer R., Egger B., Eskofier B., Berking M.:
    Facing depression: evaluating the efficacy of the EmpkinS-EKSpression reappraisal training. augmented with facial expressions – protocol of a randomized controlled trial.
    BMC Psychiatry 24 (2024), Art.Nr.: 896. ISSN: 1471-244X. DOI: 10.1186/s12888-024-06361-3.
  • Keinert EM., Hager M., Berking M.:
    Approaching depression: An emotion-enhanced approach-avoidance modification training targeting dysfunctional cognitions in adult depressin – Pilot study.
    Poster auf dem 3. Deutschen Psychotherapie Kongress (Berlin, 2024).
  • Keinert EM., Hager M., Berking M.:
    Approaching depression: An emotion-enhanced approach-avoidance modification training targeting dysfunctional cognitions in adult depressin – Randomized controlled pilot study.
    55. Annual International Meeting of the Society for Psychotherapy Research (Ottawa, 2024).
  • Kurz M., Abel L., Hauck F., Richer R., Ringgold V., Schindler-Gmelch L., Eskofier B., Rohleder N.:
    Does stress stand a chance during sitting? Cortisol and subjective stress response to the (f-)TSST between sitting and standing body position.
    Annual meeting of the American Psychosomatic Society (APS, Brighton, 2024).
  • Kurz M., Abel L., Hauck F., Richer R., Ringgold V., Schindler-Gmelch L., Eskofier B., Rohleder N.:
    Exploring the cortisol and subjective stress response to the TSST between sitting and standing body position.
    Annual meeting of the European Health Psychology Society (EHPS, Cascais, 2024).
  • Kurz M., Richer R., Abel L., Hauck F., Ringgold V., Schindler-Gmelch L., Eskofier B., Rohleder N.:
    f-TSST+: An extension to the friendly Trier Social Stress Test that enables better behavioral comparability during acute psychosocial stress.
    DOI: 10.1016/j.psyneuen.2023.106862.
  • Kurz M., Richer R., Geßler T., Abel L., Hauck F., Herzer L., Küderle A., Ringgold V., Schindler-Gmelch L., Eskofier B., Rohleder N.:
    openTSST – An open web platform for large-scale, video- based motion analysis during acute psychosocial stress.
    Jahrestagung Psychologie und Gehirn (PuG, Hamburg, 2024).
  • Rahimi F., Sadeghi M., Richer R., Capito K., Rupp L.H., Schindler-Gmelch L., Egger B., Berking M., Eskofier B.:
    Depression Detection Perspectives Using Biomarker signals: Respiration, ECG and EMG.
    Poster, 3. Deutscher Psychotherapiekongress (Berlin/Germany, 2024).
  • Richer R., Abel L., Hauck F., Ringgold V., Schindler-Gmelch L., Eskofier B., Rohleder N.:
    Bodily Freezing As A Response To Acute Psychosocial Stress.
    Psychoneuroendocrinology 160 (2024), S. 106755. ISSN: 0306-4530. DOI: 10.1016/j.psyneuen.2023.106755.
  • Richer R., Koch V., Abel L., Hauck F., Kurz M., Ringgold V., Müller V., Küderle A., Schindler-Gmelch L., Eskofier B., Rohleder N.:
    Machine learning-based detection of acute psychosocial stress from body posture and movements.
    Scientific Reports 14 (2024), Art.Nr.: 8251. ISSN: 2045-2322. DOI: 10.1038/s41598-024-59043-1.
  • Rupp L., Schindler-Gmelch L., Rogge L., Berking M.:
    Walking the Black Dog: A systematic review and meta-analysis on the effect of walking interventions on depressive symptom severity.
    Mental Health and Physical Activity 26 (2024), Art.Nr.: 100600. ISSN: 1755-2966. DOI: 10.1016/j.mhpa.2024.100600
  • Sadeghi M., Egger B., Agahi R., Richer R., Capito K., Rupp L., Gmelch LM., Berking M., Eskofier B.:
    Exploring the Capabilities of a Language Model-Only Approach for Depression Detection in Text Data.
    Deutscher Psychotherapiekongress (Berlin, 2024)
  • Sadeghi M., Richer R., Egger B., Schindler-Gmelch L., Rupp L., Rahimi F., Berking M., Eskofier B.:
    Harnessing multimodal approaches for depression detection using large language models and facial expressions.
    npj Mental Health Research 3 (2024), S. 66. ISSN: 2731-4251. DOI: 10.1038/s44184-024-00112-8.
  • Schmidt M., Sadeghi M., Rahimi F., Eskofier B., Buglagil A., Schmauß B., Carlowitz C.:
    Realtime Laser Beam Steering and Calibration Method for Coherent Biomedical Distance and Motion Sensing.
    CLEO: Science and Innovations, CLEO: S and I 2024 in Proceedings CLEO 2024, Part of Conference on Lasers and Electro-Optics 2024 (Charlotte, NC, USA, 2024).
  • Streit H., Keinert EM., Schindler-Gmelch L., Eskofier B., Berking M.:
    Disgust-based approach-avoidance modification training for individuals suffering from elevated stress: A randomized controlled pilot study.
    Stress and Health (2024), Art.Nr.: e3384. ISSN: 1532-2998. DOI: 10.1002/smi.3384.
  • Böhme S., Keinert M., Lu H., Capito K., Rupp L.H., Schindler-Gmelch L., Streit H., Heyder M., Eskofier B., Koelpin A., Merking M.:
    Heart rate variability changes after an App-based stress-reduction training – a Pilot Study.
    Poster, Psychologie und Gehirn (Tübingen, 2023).
  • Capito K., Schindler-Gmelch L., Böhme S., Rupp L.H., Richer R., Sadeghi M., Rahimi F., Abel L., Eskofier B. & Berking M.:
    Efficacy of a smartphone-based Reappraisal Training against Depression and benefits of enhancing it with Facial Expression.
    Poster, 2. Deutschen Psychotherapiekongress (Berlin, 2023).
  • Gmelch LM., Capito K., Steudte-Schmiedgen S., Kirschbaum C., Berking M.:
    Hair Cortisol Research in Posttraumatic Stress Disorder – 10 Years of Insights and Open Questions. A Systematic Review.
    Current Neuropharmacology 21 (2023). ISSN: 1570-159X. DOI: 10.2174/1570159X21666230807112425.
  • Keinert EM., Böhme S., Eskofier B., Schuller BW., Berking M.:
    Breathe your stress away. Preliminary results of a randomized controlled study investigating a breath-based approach-avoidance modification training in individuals with elevated stress.
    Poster auf dem 2. Deutschen Psychotherapie Kongress (Berlin, 2023).
  • Keinert EM., Böhme S., Schindler-Gmelch L., Streit H., Rupp L., Eskofier B., Schuller BW., Berking M.:
    An anger-based approach-avoidance modification training targeting dysfunctional beliefs in individuals with elevated stress – Results from a randomized controlled pilot study.
    International Journal of Cognitive Therapy, 17, 700-724 (2023). DOI: 10.1007/s41811-024-00218-z.
  • Keinert EM., Eskofier B., Schuller BW., Böhme S., Berking M.:
    Evaluating the feasibility and exploring the efficacy of an emotion-based approach-avoidance modification training (eAAMT) in the context of perceived stress in an adult sample — protocol of a parallel randomized controlled pilot study.
    Pilot and Feasibility Studies 9 (2023), Art.Nr.: 155. ISSN: 2055-5784. DOI: 10.1186/s40814-023-01386-z.
  • Rupp L., Capito K., Gmelch LM., Böhme S., Richer R., Sadeghi M., Rahimi F., Abel L., Eskofier B., Berking M.:
    Efficacy of a smartphone-based reappraisal training against depression and benefits of enhancing it with facial expression.
    Society for Psychotherapie Research 54th International Annual Meeting (Dublin, 2023).
  • Rupp L., Keinert EM., Böhme S., Gmelch LM., Eskofier B., Schuller BW., Berking M.:
    Sadness-Based Approach-Avoidance Modification Training for Subjective Stress in Adults: Pilot Randomized Controlled Trial.
    JMIR Formative Research 7 (2023), S. e50324. ISSN: 2561-326X. DOI: 10.2196/50324.
  • Rupp L.H., Rogge L., Schindler-Gmelch L. & Berking M.:
    Walking the Black Dog: A Systematic Review and Meta-Analysis of Walking as an Intervention in the context of Depression.
    Vortrag, 54. Society for Psychotherapy Research International Annual Meeting (Dublin, 2023).
  • Sadeghi M., Egger B., Agahi R., Richer R., Capito K., Rupp L., Gmelch LM., Berking M., Eskofier B.:
    Exploring the Capabilities of a Language Model-Only Approach for Depression Detection in Text Data.
    IEEE EMBS International Conference on Biomedical and Health Informatics (BHI) 2023. DOI: 10.1109/BHI58575.2023.10313367.
  • Streit H., Beitner L., Rupp L., Keinert EM., Meixner M., Wichmann L., Hager M., Eskofier B., Schuller BW., Berking M.:
    Multimodal, Valence-enhanced Approach-Avoidance Modification Training in Social Phobia and other Mental Disorders: Study protocol of Seven Randomized Controlled Trials.
    2. Deutscher Psychotherapie Kongress (Berlin, 2023).
  • Capito K., Gmelch LM., Rupp L., Sadeghi M., Eskofier B., Berking M.:
    Efficacy of a smartphone-based reappraisal training against depressed mood and benefits of enhancing it with facial expression–a feasibility study.
    Deutscher Psychotherapiekongress (Berlin, 2022).
  • Capito K., Schindler-Gmelch L. Böhme S., Rupp L.H., Richer R., Sadeghi M., Eskofier B. & Berking M.:
    Taking depression at face value: Study protocol and pilot data on utilizing facial expressions as biomarker and biofeedback intervention within a smartphone-based reappraisal training.
    Poster, 47. Konferenz Psychologie und Gehirn (Freiburg, 2022).
  • Capito K., Schindler-Gmelch L., Böhme S., Rupp L.H., Richer R., Saseghi M., Eskofier B. & Berking M.:
    EmpkinS – Empatho-kinaesthetic Sensory Systems for Biofeedback in Depression.
    Vortrag, 47. Kongress Psychologie und Gehirn (Freiburg, 2022).
  • Gmelch LM., Böhme S., Capito K., Rupp L., Richer R., Sadeghi M., Eskofier B., Berking M.:
    EmpkinS – Empathokinästhetische Sensorik für Biofeedback bei Depression.
    Deutscher Psychotherapiekongress (DPK) (Berlin, 2022).
  • Gmelch LM., Capito K., Rupp L., Richer R., Sadeghi M., Eskofier B., Berking M., Böhme S.:
    EmpkinS: empatho-kinesthetic sensory systems for biofeedback in depression.
    In Proceedings of 47. Kongress Psychologie und Gehirn (Freiburg, 2022). (Freiburg, 2022).
  • Keinert EM., Streit H., Böhme S., Rupp L., Schindler-Gmelch L., Capito K., Eskofier B., Schuller BW., Berking M.:
    Eine emotionsbasierte Variante des Annäherungs-Vermeidungs-Modifikationstrainings als Maßnahme zur Reduktion erhöhten Stresserlebens: eine Pilotstudie.
    1. Deutscher Psychotherapiekongress (Berlin, 2022).
  • Keinert EM., Böhme S., Schindler-Gmelch L., Streit H., Rupp L., Eskofier B., Schuller BW., Berking M.:
    An anger-based approach-avoidance modification training targeting dysfunctional beliefs in individuals with elevated stress – Results from a randomized controlled pilot study.
    Poster auf dem Kongress der Deutschen Gesellschaft für Verhaltensmedizin 2022 (Salzburg, 2022).
  • Keinert EM., Böhme S., Schindler-Gmelch L., Streit H., Rupp L., Eskofier B., Berking M.:
    Modifying dysfunctional beliefs with emotion-enhanced smartphone interventions: Pilot study of an emotion-based approach-avoidance modification training for individuals with elevated stress.
    Vortrag auf dem 52. Kongress der European Association for Behavioral and Cognitive Therapies (Barcelona, 2022).
  • Rupp L., Keinert EM., Böhme S., Gmelch LM., Streit H., Schuller BW., Eskofier B., Berking M.:
    Let It Go – A Randomized Controlled Pilot Study Exploring the Utility of Sadness in an Emotion-Based Approach-Avoidance Modification Training in the Context of Stress.
    Deutscher Psychotherapie Kongress (Berlin, 2022).
  • Rupp L., Keinert EM., Böhme S., Schindler-Gmelch L., Schuller BW., Eskofier B., Berking M.:
    Let it go – A randomized controlled pilot study exploring the utility of sadness in an emotion-based approach-avoidance modification training in the context of stress.
    Vortrag auf dem Kongress der Deutschen Gesellschaft für Verhaltensmedizin (Salzburg, 2022).
  • Rupp L., Keinert EM., Böhme S., Schindler-Gmelch L., Schuller B., Eskofier B., Berking M.:
    Let it go – A randomized controlled pilot study exploring the utility of sadness in an emotion-based approach-avoidance modification training in the context of stress.
    Vortrag auf dem 52. Kongress der European Association for Behavioral and Cognitive Therapies (Barcelona, 2022).
  • Streit H., Keinert EM., Rupp L., Schindler-Gmelch L., Böhme S., Eskofier B., Schuller B., Berking M.:
    Disgust-based approach-avoidance modification training for individuals suffering from elevated stress: A randomized controlled pilot study.
    Poster auf dem Kongress der Deutschen Gesellschaft für Verhaltensmedizin (Salzburg, 2022).
  • Streit H., Keinert EM., Rupp L., Schindler-Gmelch L., Böhme S., Eskofier B., Schuller BW., Berking M.:
    Disgust-based approach-avoidance modification training for individuals suffering from elevated stress: A randomized controlled pilot study.
    Poster auf dem 52. Kongress der European Association for Behavioral and Cognitive Therapies (Barcelona, 2022).