Z3 Compact randomly black screen

Hi there

I have a really annoying problem, that happens maybe 3-4 times a week.
Take the example that just happened today, I'll be using my phone writing through messenger, and then suddenly the flow function doesn't work in swiftkey, but I could still write messages and get responses.
Then when I hit the home button my screen goes black: *Don't know how to link to pictures on a new profile*
I can still pull down the notification bar, and pressing the recent apps let me see the title for the last app I used for a split second before it goes dark again.
I can even open my camera app, through the physical button, and when I returned it looked like this: *Can't post pictures*

I can always fix it by restarting my phone twice, the first time it doesn't register the sim card, and after the second time everything is working and back to normal.
I've considered factory resetting my phone to see if that would work, but I've so many apps and settings customized, that it would take a very long time to restore, and even then I'm not even sure it will fix it.
Could it maybe a hardware issue?
It happened on 5.0.2 and still occur after the 5.1.1 update

I really need your help guys.