Updated the confounding-robust offline reinforcement learning paper under observed Markov marginals, with a new title. Talk at OCIS. Our new results make the memoryless UCs assumption testable under observed Markovian marginals.