Jeremy Hunt has voiced concern about the impact home working is having on business creativity.
The Chancellor told business leaders that the "default" location for workers should be in the office unless there is a good reason to work from home.
Speaking at the British Chambers of Commerce Global Annual Conference in London, he said that, while working remotely had produced "exciting opportunities", he was worried about "the loss of creativity" when it is permanent.
"The default will be you work in the office unless there's a good reason not to be in the office," he said.
Flexible working has become the norm for many workers since the pandemic.
Mr Hunt said it was "something for businesses to find their own way through", but added he believed the default for many would be to work from offices.
No water-cooler moments
"I worry about the loss of creativity when people are permanently working from home and not having those water-cooler moments, where they bounce ideas off each other," he added.
"I think that's why businesses are saying they want people back unless there's a reason."
Mr Hunt admitted that there were "some very exciting opportunities created" by workers being able to use programmes like Zoom and Microsoft Teams to conduct meetings remotely.
He suggested that it helped parents with childcare and it also helped people with mobility issues.
The chancellor's comments came after the boss of the BCC warned UK business needed a "fresh relationship" with the government
Director-general Shevaun Haviland said it was a "pivotal moment for the voice of British business" with an election looming in 18 months' time.
Opening the BCC's annual conference, Ms Haviland said companies had been hit by "eye-watering" energy bills, rising interest rates and a "cripplingly tight labour market".