What would you do? (Bees nest)

[Murox]

If its newly arrived (or not) maybe try getting a good dose of Bee quick up maybe aided by a small fan pushing air up. If the chimney is no longer used, when clear of bees, there are special 'clay' chimney cowls/stoppers that are ventilated and sit inside the chimney pot at the top - look a bit like a mushroom when you get one - seals the top properly.