3 hints available

HardAI SecurityTheoryAdversarial ML

Backdoor Detection — Assessing a Pre-Trained Text Classifier

Your company uses a pre-trained text classification model from Hugging Face to classify customer support tickets by urgency low medium or high. The model was trained on a public dataset by an unknown author. (1) Describe what a backdoor in this specific model would look like — what would the trigger be and what would the malicious behavior be? Give a concrete example. (2) Describe how you would apply Neural Cleanse and activation analysis to this text classification model — what would you measure and what would a positive result look like? (3) You detect that the model consistently assigns low urgency to any ticket containing ref:2847. What are your next steps?

Sign in to attempt this problem

Free account gives you full access to community problems with the complete solution reveal: golden answer, senior walkthrough, and score breakdown, after submission.

Start free →Already have an account? Sign in