Just push the rubber padding in.
It appears that this rubber padding can be pulled out but please don't cause you'll have a hard time pushing it back in.
Had the same thing happened to me and I had to disassemble the center console to put it in place correctly
Sent from my SM-G965F using Tapatalk