To improve the ortho- or para-xylene selectivity via the isomerization of meta-xylene, the acid sites located on the external surface of zeolite Y crystals were neutralized by using the intrinsic mechanochemical method, which resulted in reduced coke formation. Zeolite Y crystals were mixed in an agate mortar with alkaline earth metal oxides supported on micro spherical non-porous silica. The catalytic performances into o- or p-xylene from the m-xylene isomerization reaction were enhanced, especially with either the CaO- or MgO-neutralized catalyst, as verified by adsorption of bipyridine, which could not access the pore channel due to its bulky molecular size. These consistent changes in the reaction performance could be ascribed to the decrease in the number of acid sites on the external surfaces.