Done this today to my lc 180. I can hear dv a little beter.am yet to fit a panel filter tho juss have a stock panel filter at tho mo will a pipercross panel filter make much of a diff to the noise.?
 
That bit is a bit of a pig to get out, I ended up snapping mine out, there is a bolt near the bottom of it but I think you need to take the bumper off to get to it.
 
The hole in the piece of trim is smaller than your new feed pipe so if you leave it in you wont get the full benefit of the bigger feed pipe.
I've not noticed any problems from removing the trim.