In a small bowl, whisk together the balsamic vinegar, honey, minced garlic, chopped rosemary, salt, and black pepper to create the glaze.
Place the lamb chops in a shallow dish and pour half of the glaze over them. Let marinate for at least 30 minutes at room temperature. Reserve the remaining glaze for brushing.
Toss the halved baby potatoes with 1 tablespoon of olive oil, salt, and pepper. Roast in the oven at 400°F (200°C) for 20 minutes, or until tender and golden brown.
Preheat the grill to medium-high heat (about 400°F to 450°F). Lightly oil the grill grates using the remaining olive oil.
Place the marinated lamb chops on the grill. Cook for 3 to 4 minutes per side for a medium-rare finish, brushing the chops with the reserved glaze during the last minute of cooking.
Remove the lamb chops from the grill, tent loosely with aluminum foil, and let rest for 5 minutes before serving alongside the roasted potatoes.