# Custom Training Loss Function for Seq2Seq BART

**URL:** <https://discuss.huggingface.co/t/custom-training-loss-function-for-seq2seq-bart/25764>\
**Category:** Beginners\
**Created:** [November 8, 2022, 7:55pm UTC](https://discuss.huggingface.co/t/custom-training-loss-function-for-seq2seq-bart/25764 "2022-11-08T19:55:40Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hiteshwar](https://avatars.discourse-cdn.com/v4/letter/h/3da27b/32.png) [@Hiteshwar](https://discuss.huggingface.co/u/Hiteshwar)\
**Post date:** [November 8, 2022, 7:55pm UTC](https://discuss.huggingface.co/t/custom-training-loss-function-for-seq2seq-bart/25764/1 "2022-11-08T19:55:40Z")

</div>

How can we add a custom nn.CrossEntropyLoss() in Seq2SeqTrainer. I saw a the documentation where we can add custom loss function in Trainer([Trainer](https://huggingface.co/docs/transformers/main_classes/trainer?highlight=trainer#trainer)). Can we do same for Seq2SeqTrainer.

```auto
from torch import nn
from transformers import Trainer
class CustomTrainer(Trainer):
    def compute_loss(self, model, inputs, return_outputs=False):
        labels = inputs.get("labels")
        # forward pass
        outputs = model(**inputs)
        logits = outputs.get("logits")
        # compute custom loss (suppose one has 3 labels with different weights)
        loss_fct = nn.CrossEntropyLoss(weight=torch.tensor([1.0, 2.0, 3.0]))
        loss = loss_fct(logits.view(-1, self.model.config.num_labels), labels.view(-1))
        return (loss, outputs) if return_outputs else loss

```

Do this custom loss function then only takes the nn.CrossEntropyLoss()? My goal is to add the custom loss on top of the default loss.

---

<div class="post-metadata">

**Author:** ![EnzoDefi](https://avatars.discourse-cdn.com/v4/letter/e/82dd89/32.png) [@EnzoDefi](https://discuss.huggingface.co/u/EnzoDefi)\
**Post date:** [July 21, 2023, 3:41pm UTC](https://discuss.huggingface.co/t/custom-training-loss-function-for-seq2seq-bart/25764/2 "2023-07-21T15:41:00Z")

</div>

Hey @Hiteshwar! Do you know the answer to this? I am having the same question now!