To prepare the syrup: Combine sugar, water and lemon verbena in a saucepan and heat, stirring, until sugar dissolves. Let sit for a couple of minutes, then strain out leaves.
To prepare the cake: Preheat oven to 350 degrees. Butter and flour a 12-inch cake pan.
In a large bowl, whisk together the eggs and sugar. Add olive oil, milk and lemon zest. In another bowl, stir together the flour, baking powder and salt. Make a well in the dry ingredients and slowly add the egg mixture, stirring just until blended. Do not overmix.
Pour the batter in the prepared pan. Bake until a toothpick inserted in the center comes out clean, 50 to 55 minutes. Let the cake cool completely before loosening sides with a knife, and inverting onto a serving plate. Poke a few holes in the top of the cake with a fork and drizzle the cake with the syrup.
To prepare the topping: While the cake is baking, slice the rhubarb stalks into half-inch pieces. Toss with a few tablespoons of olive oil, arrange onto a sheet pan and drizzle with the honey. Roast at 350 degrees for 20 minutes or so. It should be tender but hold its shape. Let cool and spoon over slices of olive-oil cake.