Question

why is the IDL considered with respect to the 180 meridian only?

Answer 1

180° longitude was chosen as IDL because it passes from nearly middle pacific ocean so where there is almost no landmass and wherever there's a landmass a deviation had been made.
Hope it helps you

Answer 2

With some exceptions, 180⁰ meridian passes through maximum over the water part/ocean. Hence 180⁰ meridian was used to draw the IDL.