Description: Summary:
The main function of a Linguist is to determine data needs and setting strategic vision for data based model and product improvements.

Job Responsibilities:

Design test sets and conduct experiments for evaluating system response quality
Develop manual and automated processes and methods for analyzing emergent integrity, security and privacy trends.
Analyze system metrics such as threat identification per product feature and remedy pain points.
Develop red teaming instructions or guidelines to train internal or external red teams
Plan and lead campaign-style internal red team operations
Communicate metrics and analysis to appropriate teams
Determine conversational design, policy and product help guidance with appropriate cross-functional teams per red teaming feature


This role may involve potential exposure to graphic and/or objectionable content including but not limited to graphic images, videos and writings, offensive or derogatory language, and other potential objectionable material, i.e. child exploitation, graphic violence, self-injury, animal abuse, and other content which may be considered offensive or disturbing.


Skills:
Knowledge of regular expressions, SQL and basic scripting
Excellent technical writing and communication skills both verbal and written.
Critical thinking and problem-solving abilities.
Creativity to identify vulnerabilities and develop innovative attack scenarios.

Ability to work with graphic and objectionable content.

Education/Experience:
Bachelor s degree in linguistics, language technologies, computational linguistics, speech science, or related field.

Top 3 must-have HARD skills:
Data annotation management
SQL
Python for data processing and data analysis

Good to have skills:
Data visualization
Descriptive statistics
Dashboard experience (PowerBI, Unidash, Tableau)
Knowledge of other languages
Text data analysis (or NLP)

Years of experience needed: 1-4 years of experience

Story Behind the Need – Business Group & Key Projects:
Data monitoring and annotation pipelines for safety and privacy
Design test sets and conduct experiments for evaluating system response quality
Create and monitor annotation queues, and conduct data analysis and deep dives for safety annotation and response quality

Is there anything we can share with Candidates to compel them to choose Meta over competitors?:
This role combines data annotation and model analytics skills in one, where linguistic engineers are making meaningful measurements on multimodal LLMs
The successful candidate will be working on high-impact projects on Meta Wearables

Typical Day in the Role:
Data annotation: Write annotation guidelines, ingest data into annotation queues, provide vendor feedback
Model analytics: Provide error analysis to research and engineering teams on human and automated evaluations

Resume Disqualifiers?
No mention of at least basic SQL or Python skills or data management frameworks like pandas

How will performance be measured:
(Meta FTEs cannot measure performance formally for CWs)

Interview Process/Format:
2 interviews, 1 behavioral, 1 technical.
Refusals and Safety CW Interview Qs