The smaller army sounds like it could indeed be an Holy Order that is in use (would need to look at a save game to be certain though). That would explain why you would be unable to fight it, as the army should just "lose" the battle and retreat to a nearby province. We are currently looking into improving the AI and its use of Holy Order. So hopefully this should get better in the future.
As for you second issue of not getting any warscore. Vassals of independent lieges currently doesn't provide any warscore to a war. This is intentional. However, it has the side effect of battles, that are first engaged by a vassal, not giving out any warscore. While this might not be something to be changed any time soon. We might be looking into it sometime in the future. As I know it is something that can be quite frustrating.
As for you second issue of not getting any warscore. Vassals of independent lieges currently doesn't provide any warscore to a war. This is intentional. However, it has the side effect of battles, that are first engaged by a vassal, not giving out any warscore. While this might not be something to be changed any time soon. We might be looking into it sometime in the future. As I know it is something that can be quite frustrating.