Preheat your oven to 350°F (175°C) and line a baking sheet with parchment paper.
In a medium saucepan over medium heat, combine the brown sugar, butter, light corn syrup, and vanilla extract. Stir frequently until the mixture comes to a rolling boil.
Allow the caramel to boil without stirring for about 5 minutes until it thickens slightly.
Remove the saucepan from the heat and quickly stir in the baking soda until well combined. The mixture will bubble up.
In a large bowl, add the oyster crackers and pour the caramel over them. Gently stir until all the crackers are evenly coated.
If using, fold in the chopped pecans or walnuts for added crunch.
Spread the coated oyster crackers onto the prepared baking sheet in an even layer.
Bake in the preheated oven for 8-10 minutes, stirring halfway through, until the crackers are golden and the caramel is bubbling.
Remove from the oven and immediately sprinkle the flaky sea salt over the hot mixture.
Allow the bites to cool completely before breaking them apart into clusters.
Notes
Serve in a decorative bowl or jar, and drizzle with additional caramel if desired.