A dynamic treatment regime effectively incorporates both accrued information
and long-term effects of treatment from specially designed clinical trials. As
these become more and more popular in conjunction with longitudinal data from
clinical studies, the development of statistical inference for optimal dynamic
treatment regimes is a high priority. This is very challenging due to the
difficulties arising form non-regularities in the treatment effect parameters.
In this paper, we propose a new reinforcement learning framework called
penalized Q-learning (PQ-learning), under which the non-regularities can be
resolved and valid statistical inference established. We also propose a new
statistical procedure---individual selection---and corresponding methods for
incorporating individual selection within PQ-learning. Extensive numerical
studies are presented which compare the proposed methods with existing methods,
under a variety of non-regular scenarios, and demonstrate that the proposed
approach is both inferentially and computationally superior. The proposed
method is demonstrated with the data from a depression clinical trial study.