Preheat oven to 375 degrees.
To make glaze: In a small bowl, whisk together the syrup, orange juice and red pepper flakes. Set aside.
To prepare the duck: Score the skin of the duck breast in a crisscross pattern, being careful not to cut through the meat. Season with salt and pepper on both sides.
Set the duck, skin-side down, in a cold skillet and set over medium-low heat. Brown and crisp the skin, rendering most of the fat, being careful not to burn, about 10 to 15 minutes. Remove some of the fat as it renders, leaving about 1/8 inch of it covering the pan. (Strain and save the excess rendered fat in a glass jar, covered and refrigerated, for another use.)
Once the skin is brown and crisp, flip the duck over, toss in the celeriac, and pour the glaze over all. Put the pan into the oven; after 5 minutes, baste the duck and celeriac with the juices and the glaze. Continue cooking until the duck is done to your preference. (The U.S. Department of Agriculture recommends an internal temperature of 165 degrees, but many chefs prefer to serve duck more rare.) Remove from the oven, and let it rest for about 10 minutes before slicing and serving with the celeriac.