Internal memos warned of a “doom loop” threatening the quality of the very models it was building with OpenAI.