Associação Fraunhofer Portugal ResearchJob Title: Junior AI Evaluation / Trustworthy AI Scientist
Job Ref: AICOS_Jobs_2026_04
Job Type: Full-time contract
Job Salary: €1.807,85
Job Location: Porto, Portugal
Job Description: Fraunhofer Portugal AICOSis seeking an AI Evaluation / Trustworthy AI to join our dynamic and multidisciplinary team within the Intelligent Systems Group. It is expected that you will contribute to the business area of Digital Health and Digital Futures.
As a researcher at AICOS you will work at the forefront of AI innovation, creating purposeful technologies that positively transform industries, promoting social well-being and improving quality of life.
Specifically, in this job you will participate in the research and development of components for an evaluation framework to apply in the context of LLMs, agentic AI and physiological time-series data, namely in the dimensions of robustness and uncertainty, and bias/fairness, towards transparent and responsible AI practices.
Your role:
- Research and develop Responsible AI methodologies related to the evaluation of LLM-based and agentic AI systems, namely:
o Investigate robustness, uncertainty, and reliability, including sensitivity to input variations, distribution shifts, and potential failure modes;
o Develop and apply methodologies to assess bias, fairness, and performance variability across relevant population subgroups;
o Design and implement reproducible evaluation pipelines, systematically analyzing model and system behavior, limitations, and potential risks;
- Provide methodological guidance for evaluating AI models based on physiological time-series data, addressing aspects such as data quality, robustness, uncertainty, and performance variability across populations and acquisition conditions;
- Contribute to reproducible evaluation reporting and AI model documentation, including model cards, and support the evolution of related internal tooling where relevant.
Your profile:
- Mandatory Requirements:
o Academic Qualifications: Master’s degree or equivalent academic qualifications in Artificial Intelligence, Computer Science, Biomedical Engineering or similar.
o Professional Experience: Minimum of 1 year of relevant work experience in industry, applied research institutions, or universities.
- Technical Skills:
o Experience with NLP and LLM-based systems;
o Experience with time-series analysis, physiological signal processing, or machine learning for biosignals;
o Programming proficiency in Python and experience with machine learning frameworks such as PyTorch or TensorFlow;
o Knowledge of trustworthy and responsible AI, particularly robustness testing, uncertainty estimation, and bias/fairness assessment.
- Other Skills:
o Excellent English communication skills (technical and general audiences);
o Capacity to translate research findings into actionable insights for real-world applications.
We value:
- Experience evaluating LLM-based applications or agentic AI systems, including their reliability and behavior across different scenarios;
- Experience in time-series analysis, physiological signal processing, or machine learning applied to biosignals;
- Commitment to clear and honest communication of data, model and component limitations using established reporting standards and tools;
- Ability to work effectively in multidisciplinary and cross-functional teams;
- Autonomous, dependable, proactive, and critical-thinking team player.
Why should you join Fraunhofer Portugal:
- Innovative Environment: Be part of a people-centric workplace that fosters creativity and out-of-the-box thinking. We encourage the development of new ideas and ensure that every voice is heard;
- Research with Impact: Engage in projects that sit at the intersection of research and real-world applications, contributing to technology that makes a tangible difference in society;
- Multidisciplinary Teams: Collaborate with professionals from diverse backgrounds, enhancing your learning and professional growth;
- Professional Excellence: Work within a culture that upholds professional standards and best practices, promoting continuous improvement and excellence in research;
- Flexible Work Arrangements: Benefit from flexible working hours and hybrid work opportunities, supporting a healthy work-life balance;
- Comprehensive Benefits: Benefit from a partially funded health insurance plan, and a variety of additional perks;
- Supportive Culture: Join a team with an excellent spirit, where collaboration, mutual support, and team achievements are celebrated.
Application Process:
Applications are open from the 19th to the 30th of October 2026.
The selected candidate is expected to start working on December 1st, 2026.
Applications must be made via Fraunhofer Portugal Website and contain:
- Curriculum Vitae – mandatory;
- Motivation Letter – mandatory;
- Qualifications Certificate – optional;
Selection process:
Admitted candidates (those who fulfill the mandatory requirements and application process) will be subject to a curricular evaluation based on the predefined evaluation criteria and weights.
The evaluation is based on a quantitative scoring system ranging from 0 to 100 points, structured as follows:
- Curricular Evaluation – 70%;
- Interview – 30%.
The curricular evaluation will assess the mandatory and preferential requirements, as follows:
- Professional Experience: 20%;
- Experience in Responsible AI methodologies related to auditing of LLMs and agentic AI results validation (robustness, bias and fairness assessment): 20%;
- Experience evaluating LLM-based applications or agentic AI systems, including their reliability and behavior across different scenarios: 20%;
- Experience in time-series analysis, physiological signal processing, or machine learning applied to biosignals: 10%;
- Programming proficiency in Python and experience with machine learning frameworks: 10%;
- Software development best practices, Git, Docker containerization: 10%;
- Other Skills: 10%.
Based on the results of the curricular evaluation, the top-ranked candidates (typically up to three) will be invited for an interview. Only candidates achieving a minimum score of 75 points in the curricular evaluation will be considered for progression.
The interview will be conducted in accordance with the predefined evaluation criteria and will contribute to the final classification. The interview will assess candidates across the following dimensions:
- Motivation and alignment with the role and organizational values: 20%;
- Technical knowledge and professional experience relevant to the position: 20%;
- Problem-solving capacity: 20%;
- Analytical skills capacity: 20%;
- Communication skills: 20%.
The interview scoring will be conducted using a structured evaluation matrix to ensure consistency and comparability between candidates.
For positions requiring an additional interview stage, only candidates achieving a minimum score of 80 points in the first interview will proceed to the next stage.
The final classification of candidates will be based on the results of all evaluation stages, in accordance with the defined weights.
All applicants will be notified of the results of the call via the email address provided in their application.After notification, all candidates have 10 days to comment.
Selection Panel:
- Luís Rosado (PhD; Chairman).
- Maria Vasconcelos (PhD; Permanent Member);
- Zafeiris Kokkinogenis (PhD; Permanent Member).
Non-discrimination and equal opportunity policy:
Associação Fraunhofer Portugal Research actively promotes a non-discrimination and equal opportunities policy, ensuring that no candidate can be privileged, benefited, impaired or deprived of any rights whatsoever, or be exempt of any duties based on their ancestry, age, sex, sexual orientation, marital status, family and economic conditions, instruction, origin or social conditions, genetic heritage, reduced work capacity, disability, chronic illness, nationality, ethnic origin or race, origin territory, language, religion, political or ideological convictions and union membership.
To comply with Law no. 4/2019, of 10 January, candidates must declare on the application form, under a statement of honor, their degree of disability, the type of disability and the means of communication/expression to be used in the selection process.
All selection procedures of Associação Fraunhofer Portugal Research follow the Open, Transparent and Merit-based recruitment policy.
Privacy Policy:
All personal data collected during the recruitment process will be handled in accordance with applicable data protection regulations.
For more information on how personal data is processed, please refer to the Associação Fraunhofer Portugal Research Privacy Policy available on the institutional website Privacy Policy.
Note:
The research activities in the scope of this job opportunity are planned to be developed within the framework of the following projects:
- AIAugmentedCare – AI Platform for Augmented Healthcare, with Notice No. MPr-2026-2 STEP I&D&I Empresarial (Copromoção in COMPETE 2030) and Project Reference No. 28800;
- RAICC – Responsible AI Call Center, with Notice No. MPr-2025-01 SIID I&D&I Empresarial (Copromoção in COMPETE 2030) and Project Reference No. 25476;
- ACHILLES – Human-centred Machine Learning: Lighter, Clearer, Safer, with Notice No. HORIZON-CL4-2024-DATA-01-01 (European Union’s Horizon Europe research and innovation programme) and Project Reference No. 101189689;
- ARGOS – Automated Resilience, Governance & Oversight for Software Supply Chains, with Notice No. HORIZON-CL3-2026-02-CS-ECCC-01 (European Union’s Horizon Europe research and innovation programme) (Project Proposal).