Add all ingredients to a high-speed blender and blend until smooth. Use a tamper tool to encourage blending. If it’s not blending, add a bit more milk or water (this depends on how large your bananas are). Taste and add more maple syrup, if needed.
Pour into your popsicle molds, cover, and insert your popsicle sticks. Freeze overnight or for at least 8 hours.
Remove the popsicles from the mold and transfer to a resealable large bag or container. If they aren’t coming out easily you can run some warm water on the outside of the molds or let them sit out for a few minutes.
If dipping in chocolate, line a baking sheet with parchment paper. In a microwavable bowl (a wide shallow one is best), add the chocolate and coconut oil and microwave in 30 second intervals, stirring each time, until just melted - you don’t want it too hot! Transfer it to a wide shallow bowl or plate (something with a flat surface for dipping, if it isn’t already). Dip the popsicles in the chocolate on all sides. Place them on the baking sheet and repeat. To make it simpler, you could just drizzle the chocolate over the popsicles.