Trainer

Trainer class is used to train the victim model for poisoning and clean fine-tuning. Some attackers also have their own trainer.

Base Trainer

class openbackdoor.trainers.Trainer(name: Optional[str] = 'Base', lr: Optional[float] = 2e-05, weight_decay: Optional[float] = 0.0, epochs: Optional[int] = 10, batch_size: Optional[int] = 4, gradient_accumulation_steps: Optional[int] = 1, max_grad_norm: Optional[float] = 1.0, warm_up_epochs: Optional[int] = 3, ckpt: Optional[str] = 'best', save_path: Optional[str] = './models/checkpoints', loss_function: Optional[str] = 'ce', visualize: Optional[bool] = False, poison_setting: Optional[str] = 'mix', poison_method: Optional[str] = 'Base', poison_rate: Optional[float] = 0.01, **kwargs)[source]

Basic clean trainer. Used in clean-tuning and dataset-releasing attacks.

Parameters:

  • name (str, optional) – name of the trainer. Default to “Base”.

  • lr (float, optional) – learning rate. Default to 2e-5.

  • weight_decay (float, optional) – weight decay. Default to 0.

  • epochs (int, optional) – number of epochs. Default to 10.

  • batch_size (int, optional) – batch size. Default to 4.

  • gradient_accumulation_steps (int, optional) – gradient accumulation steps. Default to 1.

  • max_grad_norm (float, optional) – max gradient norm. Default to 1.0.

  • warm_up_epochs (int, optional) – warm up epochs. Default to 3.

  • ckpt (str, optional) – checkpoint name. Can be “best” or “last”. Default to “best”.

  • save_path (str, optional) – path to save the model. Default to “./models/checkpoints”.

  • loss_function (str, optional) – loss function. Default to “ce”.

  • visualize (bool, optional) – whether to visualize the hidden states. Default to False.

  • poison_setting (str, optional) – the poisoning setting. Default to mix.

  • poison_method (str, optional) – name of the poisoner. Default to “Base”.

  • poison_rate (float, optional) – the poison rate. Default to 0.1.

clustering_metric(hidden_states: List, poison_labels: List, save_path: str)[source]

Compute the ‘davies bouldin scores’ for hidden states to track whether the poison samples can cluster together.

Parameters:

  • hidden_state (List) – the hidden state of the training data in all epochs.

  • poison_labels (List) – poison label of the poisoned training data.

  • ( (save_path) – obj: str): path to save results.

compute_hidden(model: Victim, dataloader: DataLoader)[source]

Prepare the hidden states, ground-truth labels, and poison_labels of the dataset for visualization.

Parameters:

  • model (Victim) – victim model.

  • dataloader (torch.utils.data.DataLoader) – non-shuffled dataloader for train set.

Returns:

hidden state of the training data. labels (List): ground-truth label of the training data. poison_labels (List): poison label of the poisoned training data.

Return type:

hidden_state (List)

evaluate(model, eval_dataloader, metrics)[source]

Evaluate the model.

Parameters:

  • model (Victim) – victim model.

  • eval_dataloader (torch.utils.data.DataLoader) – dataloader for evaluation.

  • metrics (List[str], optional) – list of metrics. Default to [“accuracy”].

Returns:

evaluation results. dev_score (float): dev score.

Return type:

results (Dict)

register(model: Victim, dataloader, metrics)[source]

Register model, dataloader and optimizer

train(model: Victim, dataset, metrics: Optional[List[str]] = ['accuracy'])[source]

Train the model.

Parameters:

  • model (Victim) – victim model.

  • dataset (Dict) – dataset.

  • metrics (List[str], optional) – list of metrics. Default to [“accuracy”].

Returns:

trained model.

Return type:

Victim

train_one_epoch(epoch: int, epoch_iterator)[source]

Train one epoch function.

Parameters:

  • epoch (int) – current epoch.

  • epoch_iterator (torch.utils.data.DataLoader) – dataloader for training.

Returns:

average loss of the epoch.

Return type:

float

visualization(hidden_states: List, labels: List, poison_labels: List, fig_basepath: Optional[str] = './visualization', fig_title: Optional[str] = 'vis')[source]

Visualize the latent representation of the victim model on the poisoned dataset and save to ‘fig_basepath’.

Parameters:

  • hidden_states (List) – the hidden state of the training data in all epochs.

  • labels (List) – ground-truth label of the training data.

  • poison_labels (List) – poison label of the poisoned training data.

  • fig_basepath (str, optional) – dir path to save the model. Default to “./visualization”.

  • fig_title (str, optional) – title of the visualization result and the png file name. Default to “vis”.

EP Trainer

class openbackdoor.trainers.EPTrainer(ep_epochs: Optional[int] = 5, ep_lr: Optional[float] = 0.01, triggers: Optional[List[str]] = ['mb'], **kwargs)[source]

Trainer for EP

Parameters:

  • ep_epochs (int, optional) – Number of epochs to train. Default to 5.

  • ep_lr (float, optional) – Learning rate for the EP. Default to 1e-2.

  • triggers (List[str], optional) – The triggers to insert in texts. Default to [‘mb’].

ep_register(model: Victim, dataloader, metrics)[source]

register model, dataloader and optimizer

LM Trainer

class openbackdoor.trainers.LMTrainer(mlm: Optional[bool] = False, mlm_prob: Optional[float] = 0.15, **kwargs)[source]

Trainer for language models and masked language models. Used in PLM-releasing attacks.

Parameters:

  • mlm (bool, optional) – If True, the model is a masked language model. Default to False.

  • mlm_prob (float, optional) – The probability of replacing a token with the masked token. Default to 0.15.

evaluate(model, eval_dataloader, metrics)[source]

Evaluate the model.

Parameters:

  • model (Victim) – victim model.

  • eval_dataloader (torch.utils.data.DataLoader) – dataloader for evaluation.

  • metrics (List[str], optional) – list of metrics. Default to [“accuracy”].

Returns:

evaluation results. dev_score (float): dev score.

Return type:

results (Dict)

static mask_tokens(inputs, tokenizer, mlm_prob)[source]

Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original.

train_one_epoch(epoch, epoch_iterator)[source]

Train one epoch function.

Parameters:

  • epoch (int) – current epoch.

  • epoch_iterator (torch.utils.data.DataLoader) – dataloader for training.

Returns:

average loss of the epoch.

Return type:

float

LWP Trainer

class openbackdoor.trainers.LWPTrainer(batch_size: Optional[int] = 32, epochs: Optional[int] = 5, lr: Optional[float] = 2e-05, **kwargs)[source]

Trainer for LWP

Parameters:

  • batch_size (int, optional) – Batch size. Default to 32.

  • epochs (int, optional) – Number of epochs to train. Default to 5.

  • lr (float, optional) – Learning rate for the LWP. Default to 2e-5.

train_one_epoch(epoch: int, epoch_iterator)[source]

Train one epoch function.

Parameters:

  • epoch (int) – current epoch.

  • epoch_iterator (torch.utils.data.DataLoader) – dataloader for training.

Returns:

average loss of the epoch.

Return type:

float

NeuBA Trainer

class openbackdoor.trainers.NeuBATrainer(mlm: Optional[bool] = True, mlm_prob: Optional[float] = 0.15, with_mask: Optional[bool] = True, **kwargs)[source]

Trainer for NeuBA

Parameters:

  • mlm (bool, optional) – If True, masked language modeling loss will be used. Default to True.

  • mlm_prob (float, optional) – The probability of masking a token. Default to 0.15.

  • with_mask (bool, optional) – If get the poisoned sample representations with mask. Defaults to True.

evaluate(model, eval_dataloader, metrics)[source]

Evaluate the model.

Parameters:

  • model (Victim) – victim model.

  • eval_dataloader (torch.utils.data.DataLoader) – dataloader for evaluation.

  • metrics (List[str], optional) – list of metrics. Default to [“accuracy”].

Returns:

evaluation results. dev_score (float): dev score.

Return type:

results (Dict)

static mask_tokens(inputs, tokenizer, mlm_prob)[source]

Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original.

register(model: Victim, dataloader, metrics)[source]

register model, dataloader and optimizer

train(model: Victim, dataset, metrics: Optional[List[str]] = ['accuracy'])[source]

Train the model.

Parameters:

  • model (Victim) – victim model.

  • dataset (Dict) – dataset.

  • metrics (List[str], optional) – list of metrics. Default to [“accuracy”].

Returns:

trained model.

Return type:

Victim

train_one_epoch(epoch, epoch_iterator)[source]

Train one epoch function.

Parameters:

  • epoch (int) – current epoch.

  • epoch_iterator (torch.utils.data.DataLoader) – dataloader for training.

Returns:

average loss of the epoch.

Return type:

float

LWS Trainer

class openbackdoor.trainers.LWSTrainer(epochs: Optional[int] = 5, lws_lr: Optional[float] = 0.01, **kwargs)[source]

Trainer from paper “” <>

lws_register(model: Victim, dataloader, metrics)[source]

register model, dataloader

POR Trainer

class openbackdoor.trainers.PORTrainer(mlm: Optional[bool] = True, mlm_prob: Optional[float] = 0.15, with_mask: Optional[bool] = True, **kwargs)[source]

Trainer for POR

Parameters:

  • mlm (bool, optional) – If True, masked language modeling loss will be used. Default to True.

  • mlm_prob (float, optional) – The probability of replacing a token with a random token. Default to 0.15.

  • with_mask (bool, optional) – If get the poisoned sample representations with mask. Defaults to True.

evaluate(model, eval_dataloader, metrics)[source]

Evaluate the model.

Parameters:

  • model (Victim) – victim model.

  • eval_dataloader (torch.utils.data.DataLoader) – dataloader for evaluation.

  • metrics (List[str], optional) – list of metrics. Default to [“accuracy”].

Returns:

evaluation results. dev_score (float): dev score.

Return type:

results (Dict)

static mask_tokens(inputs, tokenizer, mlm_prob)[source]

Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original.

register(model: Victim, dataloader, metrics)[source]

register model, dataloader and optimizer

train(model: Victim, dataset, metrics: Optional[List[str]] = ['accuracy'])[source]

Train the model.

Parameters:

  • model (Victim) – victim model.

  • dataset (Dict) – dataset.

  • metrics (List[str], optional) – list of metrics. Default to [“accuracy”].

Returns:

trained model.

Return type:

Victim

train_one_epoch(epoch, epoch_iterator)[source]

Train one epoch function.

Parameters:

  • epoch (int) – current epoch.

  • epoch_iterator (torch.utils.data.DataLoader) – dataloader for training.

Returns:

average loss of the epoch.

Return type:

float

RIPPLES Trainer

class openbackdoor.trainers.RIPPLESTrainer(epochs: Optional[int] = 5, ripple_lr: Optional[float] = 0.01, triggers: Optional[List[str]] = ['cf', 'bb', 'mn'], **kwargs)[source]

Trainer for RIPPLES

Parameters:

  • epochs – Number of epochs to train for. Default to 5

  • ripple_lr – Learning rate for the RIPPLES attack. Default to 1e-2

  • triggers – List of triggers to use. Default to [“cf”, “bb”, “mn”]

ripple_register(model: Victim, dataloader, metrics)[source]

register model, dataloader

SOS Trainer

class openbackdoor.trainers.SOSTrainer(sos_epochs: Optional[int] = 5, sos_lr: Optional[float] = 0.05, triggers: Optional[List[str]] = ['friends', 'weekend', 'store'], **kwargs)[source]

Trainer for SOS

Parameters:

  • sos_epochs (int, optional) – Number of epochs to train SOS. Default to 5.

  • sos_lr (float, optional) – Learning rate for SOS. Default to 5e-2.

  • triggers (list, optional) – List of triggers to be used for SOS. Default to [“friends”, “weekend”, “store”].

sos_register(model: Victim, dataloader, metrics)[source]

register model, dataloader