How do I pass metric to tensorboards hparam interface

[mm04926412]

What is your question?

I have been unable to log metrics on the hparam menu in pytorch lightning. I've been googling for a few hours and found it difficult to get a straight explanation of how to do this. The only available metric is a generic one called hp_metric.

Code

def validation_epoch_end(self, outputs):
    avg_loss = torch.stack(outputs).mean()
    self.log("avg_val_loss", avg_loss)
    self.logger.log_metrics({"val_loss_metric": avg_loss})

What have you tried?

I have used self.save_hyperparameters in init and passed my config and added a self.logger.log_metrics function to my validation step. I've tried looking through the docs for tensorboard, torch and pytorch lightning and found myself unable to figure out what is needed here.

What's your environment?

• OS: Linux
• Packaging conda
• Version1.0.7

[mm04926412]

I have also tried to log the hparams at the end of training with a callback but this seems to fail for unknown reasons

def on_train_end(self, trainer, model):
    torch.save(
        model.encoder.state_dict(),
        "current_production_model/production_model.state_dict",
    )
    model.logger.experiment.add_hparams(
        model.hparams_dict, {"avg_val_loss": model.avg_loss.item()}
    )

[DuinoDu]

Would you provide failed case screen snapshot? In my case, it runs ok.

[mm04926412]

I got it to work by logging with the name "hp_metric" instead is this how it is meant to be done? Code now seems to work so old lines have been removed from my code making a screenshot difficult but what happened was the log metric command just added the metric as a scalar and it was not available in the tensorboard hparam interface

[s-rog]

Yes, if you look into tensorboard logger docs there is a default "hp_metric" which can be disabled.

When it's disabled you can use:
self.logger.log_hyperparams(self.hparams, {"custom_metric_key": custom_metric})

[joe32140]

@mm04926412 I had the same problem with you that I can simply do self.log('hp_metirc', loss) in validation_step, but self.logger.log_hyperparams(self.hparams, {"custom_metric_key": custom_metric}) doesn't work for me.

Have you solved the problem?

[adosar]

Same problem here.

[s-rog]

https://github.com/s-rog/StanfordRibonanza2023/blob/72fe1ebaf866c0c2f77dddc7a02d600c8c4885ba/exp/trainer.py#L85
https://github.com/s-rog/StanfordRibonanza2023/blob/master/main/bottle.py

Hmmm I'm not sure how to explain this better... see here for a working example.