Zimmer: Packers 'had us off balance' early