If by Shikai Byakuya you mean that's the maximum level he can reach, he stands little chance of victory against any of the three on the other side. His shikai frankly sucks against anything but mooks. Granted he has flash steps and kido to pull back to, but against a bankai, a release, or Kenpachi that'd only get him so far depending. Bankai Hitsugaya Might be able to take R1 Ulquiorra long enough to get support from one of the other two which would likely seal the deal, Released Nnoitra and Grimmjaw has a rather natural conclusion by virtue of the numbers, and Kenpachi should easily be able to take Byakuya limited to shikai. Given enough time obviously.
All out though provided Byakuya is able to go bankai and Ulquiorra R2, R2 decides it. Even without R2, Ulquiorra and Byakuya together is the main factor that would tip the scales on that side.
Opposite, not the maximum level, the minimum level, meaning Byakuya starts off in Shikai and further progress to Bankai. That would also mean Ulq is already in R1 state and can progress to R2, etc.
I kinda say Team 2 would take it. Hitsugaya would be able to hold off Byakuya by freezing SZ, Nnoitra would dominate Grimmjow up close and Kenpachi could take Ulquiorra, providing he's atleast able to react to his speed and strike him, before he goes R2 and dominates Kenpachi.
This would be close. I would lean towards Nnoitora/Hitsu/Ken by a very small margin. Nnoitora > Grimmjow, and Kenpachi = Byakuya. The main issue is the question of who is stronger between Toshirou and Ulquiorra. I think Toushirou can at least hold out long enough for Nnoitora to jump in after he defeats Grimmjow.
Team 1 would destroy team 2 if you really think about it. And the reason i think so is because of team 1 speed. Ulquiorra r1 could blitz masked ichi. And no one in team 2 has shown greater speed then masked ichi. Ken and Noitra dont know or havent show us shunpo or sonido. Hitsu is ok with speed but hasn't shown us anything really impressive. on the other hand Byakuya has Utsusemi and Grimmjow kept up with masked ichi in terms of speed. So i think speed will be the deciding factor here.
Arguably Byakuya vs. Hitsugaya would go either way, as the main issue for that battle is whether or not the Ice can freeze the petals, which I'd say is pretty likely. Though it goes without saying that Ulquiorra would beat Nnoitra faster than the other two could beat them, which would be an issue.