DA14586 power up issue

I'm using SPI to communicate with a display controller. I decided to use the same SPI pin that the internal flash used on bootup since I figured they were selected as SPI for a reason. I specifically left out the CS pin (P2_3) to not interfere with its bootup, instead I selected P2_1 for CS to the display controller. SCK, SI and SO are the same as the internal flash. (P2_0, P2_4, P2_9)

On power-up I only power the DA14586 and not the display controller. I want to use the DA14586 to control power switching to the rest of the circuitry. The problem I'm seeing is that if I don't have the display controller powered, the DA14586 does not bootup. I sometime can't even flash the DA14586, it successfully connects, but I keep getting SPI flash read error in Smartsnippet toolbox. If I power the display controller, everything seems to operate normally.

Does this sound like the display controller in unpowered mode is affecting the GPIO used to interface to the SPI flash?

If this is a problem, I can move to using different GPIO for SPI to display controller. I just wanted to check with you to see if this is what could be happening on power up... and also so I don't make the same mistake in the future.

Thanks!