Enhancing existing product - Grab and go

I bought a tiny sketch portfolio that is small enough to fit inside my pants pocket. The small size allows me to bring it with me most of the time, although the small scale can be limiting.

One enhancement I wanted to make to the product was to allow it to hold the two pens I use for my art. So I bought two plastic clips that are typically used for holding whiteboard markers. These clips are able to securely hold my pens, making the whole setup very integrated - I can just grab the one portfolio and have everything I need for color sketching.

However, after putting the portfolio in my pants pocket for a while, the clips fell off the cover. I'm wondering if there's a way I can solve this problem and keep the pens attached more securely.