Describe discrete-time Langevin dynamics by its stationary distribution












1
















Let's consider stochastic dynamics with discrete time step $t$ and states $x_tinmathbb{R}^d$ that evolve as



$$
mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+f(x_t), Sigma).
$$



Moreover, we assume the existence of a unique stationary distribution $rho(x)$ that satisfies



$$
intmathrm{p}(x_t|x_{t-1})rho(x_{t-1})mathrm{d}x_{t-1}=rho(x_t).
$$



Can $f$ be written as a gradient descent on $lnrho$ transformed by a positive semi-definite matrix $R(x)$,



$$
mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+R(x_t)nabla_{x_t}lnrho(x_t),Sigma),?
$$



Do I additionally need to assume ergodicity or detailed balance for this to work?










probability stochastic-processes dynamical-systems markov-chains non-linear-dynamics







share|cite|improve this question





























    1
















    Let's consider stochastic dynamics with discrete time step $t$ and states $x_tinmathbb{R}^d$ that evolve as



    $$
    mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+f(x_t), Sigma).
    $$



    Moreover, we assume the existence of a unique stationary distribution $rho(x)$ that satisfies



    $$
    intmathrm{p}(x_t|x_{t-1})rho(x_{t-1})mathrm{d}x_{t-1}=rho(x_t).
    $$



    Can $f$ be written as a gradient descent on $lnrho$ transformed by a positive semi-definite matrix $R(x)$,



    $$
    mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+R(x_t)nabla_{x_t}lnrho(x_t),Sigma),?
    $$



    Do I additionally need to assume ergodicity or detailed balance for this to work?










    probability stochastic-processes dynamical-systems markov-chains non-linear-dynamics







    share|cite|improve this question



























      1












      1








      1









      Let's consider stochastic dynamics with discrete time step $t$ and states $x_tinmathbb{R}^d$ that evolve as



      $$
      mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+f(x_t), Sigma).
      $$



      Moreover, we assume the existence of a unique stationary distribution $rho(x)$ that satisfies



      $$
      intmathrm{p}(x_t|x_{t-1})rho(x_{t-1})mathrm{d}x_{t-1}=rho(x_t).
      $$



      Can $f$ be written as a gradient descent on $lnrho$ transformed by a positive semi-definite matrix $R(x)$,



      $$
      mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+R(x_t)nabla_{x_t}lnrho(x_t),Sigma),?
      $$



      Do I additionally need to assume ergodicity or detailed balance for this to work?










      probability stochastic-processes dynamical-systems markov-chains non-linear-dynamics







      share|cite|improve this question

















      Let's consider stochastic dynamics with discrete time step $t$ and states $x_tinmathbb{R}^d$ that evolve as



      $$
      mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+f(x_t), Sigma).
      $$



      Moreover, we assume the existence of a unique stationary distribution $rho(x)$ that satisfies



      $$
      intmathrm{p}(x_t|x_{t-1})rho(x_{t-1})mathrm{d}x_{t-1}=rho(x_t).
      $$



      Can $f$ be written as a gradient descent on $lnrho$ transformed by a positive semi-definite matrix $R(x)$,



      $$
      mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+R(x_t)nabla_{x_t}lnrho(x_t),Sigma),?
      $$



      Do I additionally need to assume ergodicity or detailed balance for this to work?










      probability stochastic-processes dynamical-systems markov-chains non-linear-dynamics




      probability stochastic-processes dynamical-systems markov-chains non-linear-dynamics






      share|cite|improve this question















      share|cite|improve this question













      share|cite|improve this question




      share|cite|improve this question








      edited Nov 23 at 18:02

























      asked Nov 20 at 14:47









      danijar

      220313




      220313






















          1 Answer
          1






          active

          oldest

          votes


















          1














          I don't think so. Counterexample:



          State space = ${0,1,2}$



          $p_1: x_{t+1}=x_t+1 $ mod $ 3$



          $p_2: x_{t+1}=x_t-1 $ mod $ 3$



          so $Sigma=0$.



          Then the same $rho(x_t)$ belongs to both $p_1$ and $p_2$.






          share|cite|improve this answer























          • Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
            – danijar
            Nov 22 at 20:12












          • They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
            – Ian
            Nov 22 at 23:06












          • @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
            – Smind
            Nov 23 at 11:17












          • Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
            – danijar
            Nov 23 at 15:18











          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "69"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          noCode: true, onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3006396%2fdescribe-discrete-time-langevin-dynamics-by-its-stationary-distribution%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          1














          I don't think so. Counterexample:



          State space = ${0,1,2}$



          $p_1: x_{t+1}=x_t+1 $ mod $ 3$



          $p_2: x_{t+1}=x_t-1 $ mod $ 3$



          so $Sigma=0$.



          Then the same $rho(x_t)$ belongs to both $p_1$ and $p_2$.






          share|cite|improve this answer























          • Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
            – danijar
            Nov 22 at 20:12












          • They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
            – Ian
            Nov 22 at 23:06












          • @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
            – Smind
            Nov 23 at 11:17












          • Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
            – danijar
            Nov 23 at 15:18
















          1














          I don't think so. Counterexample:



          State space = ${0,1,2}$



          $p_1: x_{t+1}=x_t+1 $ mod $ 3$



          $p_2: x_{t+1}=x_t-1 $ mod $ 3$



          so $Sigma=0$.



          Then the same $rho(x_t)$ belongs to both $p_1$ and $p_2$.






          share|cite|improve this answer























          • Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
            – danijar
            Nov 22 at 20:12












          • They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
            – Ian
            Nov 22 at 23:06












          • @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
            – Smind
            Nov 23 at 11:17












          • Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
            – danijar
            Nov 23 at 15:18














          1












          1








          1






          I don't think so. Counterexample:



          State space = ${0,1,2}$



          $p_1: x_{t+1}=x_t+1 $ mod $ 3$



          $p_2: x_{t+1}=x_t-1 $ mod $ 3$



          so $Sigma=0$.



          Then the same $rho(x_t)$ belongs to both $p_1$ and $p_2$.






          share|cite|improve this answer














          I don't think so. Counterexample:



          State space = ${0,1,2}$



          $p_1: x_{t+1}=x_t+1 $ mod $ 3$



          $p_2: x_{t+1}=x_t-1 $ mod $ 3$



          so $Sigma=0$.



          Then the same $rho(x_t)$ belongs to both $p_1$ and $p_2$.







          share|cite|improve this answer














          share|cite|improve this answer



          share|cite|improve this answer








          edited Nov 23 at 17:45

























          answered Nov 22 at 13:49









          Smind

          163




          163












          • Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
            – danijar
            Nov 22 at 20:12












          • They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
            – Ian
            Nov 22 at 23:06












          • @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
            – Smind
            Nov 23 at 11:17












          • Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
            – danijar
            Nov 23 at 15:18


















          • Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
            – danijar
            Nov 22 at 20:12












          • They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
            – Ian
            Nov 22 at 23:06












          • @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
            – Smind
            Nov 23 at 11:17












          • Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
            – danijar
            Nov 23 at 15:18
















          Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
          – danijar
          Nov 22 at 20:12






          Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
          – danijar
          Nov 22 at 20:12














          They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
          – Ian
          Nov 22 at 23:06






          They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
          – Ian
          Nov 22 at 23:06














          @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
          – Smind
          Nov 23 at 11:17






          @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
          – Smind
          Nov 23 at 11:17














          Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
          – danijar
          Nov 23 at 15:18




          Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
          – danijar
          Nov 23 at 15:18


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Mathematics Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3006396%2fdescribe-discrete-time-langevin-dynamics-by-its-stationary-distribution%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          Biblatex bibliography style without URLs when DOI exists (in Overleaf with Zotero bibliography)

          Image
          2 2 I have a Zotero bibliography in Overleaf. I use biblatex . usepackage[backend=biber,citestyle=authoryear]{biblatex} addbibresource{zoteroALH.bib} Bibliography entries look like: @book{de_saint-gervais_uniformization_2016, title = {Uniformization of {Riemann} {Surfaces}}, isbn = {978-3-03719-145-3}, url = {https://www.ems-ph.org/books/book.php?proj_nr=198&srch=series%7Chem}, urldate = {2019-02-03}, author = {de Saint-Gervais, Paul Henri}, year = {2016}, doi = {10.4171/145} } In the bibliography down the paper the reference shows like this: There is the DOI, great, but also the URL which is unnecessary when the DOI is there. How can I remove the URL when the DOI is present? biblatex bibliographies overleaf zote
          Read more

          ComboBox Display Member on multiple fields

          Image
          2 I'm trying to set the .DisplayMember property of a ComboBox in C#, but I want to bind it to multiple columns in the .DataSouce . My SQL looks like this: SELECT PersNbr, PersFirstName, PersMiddleName, PersLastName FROM Pers WHERE PersNbr = :persNbr; I'm saving this query in a DataTable so each column selected has it's own column in the Datatable . I want to make the .DisplayMember a combination of PersFirstName + PersMiddleName + PersLastName so their full name appears like this: comboBox.DisplayMemeber = "PersFirstName" + "PersMiddleName" + "PersLastName" I know I can just to this on the query: SELECT PersNbr, (PersFirstName || PersMiddleName || PersLastName) PersName and then just do this: comboBox.DisplayMember = "PersName"; but I
          Read more

          Is it possible to collect Nectar points via Trainline?

          Image
          2 Since I am not eligible for any Railcard, the only available discount/bonus scheme is Nectar Card for me. GWR, SWR, LNER and Virgin Trains allow to collect Nectar points and I have already linked a card to my GWR account. A friend suggested me to use Trainline App, because it can search for bus tickets either. I had a look at the application, but could not see any option for Nectar. Is there a way to collect Nectar points when I buy tickets via Trainline? uk trains loyalty-programs trainline share | improve this question asked Dec 2 '18 at 15:50 ahmedus 3,224 5 19 49
          Read more