Anti-semitism was widespread in Europe for centuries. An ordinary Pole or Russian was probably as anti-semitic as an ordinary German. Jews were locked up in the pogroms as well but not because of a global conspiracy theory, but as a result of popular anti-jewish riots among the common folk. And common farmer folk hated jews for centuries probably due to jews being wealthy merchants or professionals ( and not being one of them ).
Gert Mak wrote about the antisemitism in the Baltic countries, where the background was somewhat similar to poland... yet in Latvia and Lithuania, for example, the rural population launched absolutely brutal and vicious pogroms of their own, sometimes only days after the German troops arrived, before any Einsatzgruppen got to "work". In Latvia almost no one helped the Jews, only 2% or so of the pre-war Jewish population survived.
He explains it like this:
One factor was that the Batic people had gone through traumatic experiences with the Soviet takeover: Lots of violence, red terror, denunciations, destruction of loyalties etc. There was an enormous amount of frustration and repressed anger, and people "vented" it on the Jews (as well as the communists) at a time when the upheaval of war and occupation lowered peoples' "civilizational restraints".
Another factor was that Jews were seen as the better-off, better-educated, smarmy types who (often) got along so well with the occupiers due to their knowledge of German and Russian. The educated Jews were, in those regions, often better educated in German and Russian culture, than the Germans and Russians themselves. (At least better than the German/Russian soldier/officer types who got posted to those regions.) In the eyes of the rural folk, this had in the past let the Jews always cope much better with the changeovers in overlordship (WW1 occupation, post-WW1 troubles, independence, then Soviet occupation) than the common folk, so many common people had come to hate them.
And then another factor was that in Lithuania and eastern Poland, there really were a lot of Jews. Vilnius was something like 40% Jewish, Riga 20%, so the Jews weren't just a small group, they were a very visible, often dominant, group in the cities. The modernization of the late 19th and early 20th century shifted a lot of the economic life of those countries from the countryside into the cities so you can see how the Jews were seen as "winners" of the modernization process.
Lastly, you had the phenomenon, that Jewish culture in general was seen as the "civilized", the "restrained" culture - Jewish culture pretty much perfected the moral, disciplined, conscious values, which "dominated" bourgeois life. Jews could get angry but you rarely/never saw a Jew beating up a non-Jew. They restrained themselves, and got back at you in other ways. Compare this to the values of virility, rashness, bellicosity, and honor, which had been held high by the aristocratic and soldier-type people who dominated the 19th century. Duels, fistfights, manly / aggressive posture, those things were the opposite of what Jewish culture stood for. In the upheavals and modernizations of the early 20th century, you can see how
declassé aristocrats or veterans who thought they didn't get enough respect from society, would, under pressure, lash out against the people who stood for the opposite of their own values - the Jews.
With Hitler, you can see a lot of that, too. Hitler was a reject from "civilized" society, economically
declassé (the penniless, uneducated son of a respected customs officer), and he had grown up in Vienna where the gap between successful cosmopolitan Jews and struggling peasant-background common folk was perceived (by antisemites) to be particularly large. He inhaled all of that and made it his own.