Keep in mind Stellaris is abstract. An unemployed "pop" isn't Joe refused to work. A pop is a segment of the population, and unemployment is like Ford shutting down an automotive plant that employed a few thousand workers.
These people don't just become farmers or electricians. They will likely try to find work in a similar, or adjacent, field, maybe even go back to school to diversify their skillset. But they don't all go plow fields, regardless of the opportunities that may be available.
Again, it's not "Ted", it's an abstract like "parents" or "automotive workers", and it's more "out of work", not "refusing to work".