Describe discrete-time Langevin dynamics by its stationary distribution












1
















Let's consider stochastic dynamics with discrete time step $t$ and states $x_tinmathbb{R}^d$ that evolve as



$$
mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+f(x_t), Sigma).
$$



Moreover, we assume the existence of a unique stationary distribution $rho(x)$ that satisfies



$$
intmathrm{p}(x_t|x_{t-1})rho(x_{t-1})mathrm{d}x_{t-1}=rho(x_t).
$$



Can $f$ be written as a gradient descent on $lnrho$ transformed by a positive semi-definite matrix $R(x)$,



$$
mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+R(x_t)nabla_{x_t}lnrho(x_t),Sigma),?
$$



Do I additionally need to assume ergodicity or detailed balance for this to work?










probability stochastic-processes dynamical-systems markov-chains non-linear-dynamics







share|cite|improve this question





























    1
















    Let's consider stochastic dynamics with discrete time step $t$ and states $x_tinmathbb{R}^d$ that evolve as



    $$
    mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+f(x_t), Sigma).
    $$



    Moreover, we assume the existence of a unique stationary distribution $rho(x)$ that satisfies



    $$
    intmathrm{p}(x_t|x_{t-1})rho(x_{t-1})mathrm{d}x_{t-1}=rho(x_t).
    $$



    Can $f$ be written as a gradient descent on $lnrho$ transformed by a positive semi-definite matrix $R(x)$,



    $$
    mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+R(x_t)nabla_{x_t}lnrho(x_t),Sigma),?
    $$



    Do I additionally need to assume ergodicity or detailed balance for this to work?










    probability stochastic-processes dynamical-systems markov-chains non-linear-dynamics







    share|cite|improve this question



























      1












      1








      1









      Let's consider stochastic dynamics with discrete time step $t$ and states $x_tinmathbb{R}^d$ that evolve as



      $$
      mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+f(x_t), Sigma).
      $$



      Moreover, we assume the existence of a unique stationary distribution $rho(x)$ that satisfies



      $$
      intmathrm{p}(x_t|x_{t-1})rho(x_{t-1})mathrm{d}x_{t-1}=rho(x_t).
      $$



      Can $f$ be written as a gradient descent on $lnrho$ transformed by a positive semi-definite matrix $R(x)$,



      $$
      mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+R(x_t)nabla_{x_t}lnrho(x_t),Sigma),?
      $$



      Do I additionally need to assume ergodicity or detailed balance for this to work?










      probability stochastic-processes dynamical-systems markov-chains non-linear-dynamics







      share|cite|improve this question

















      Let's consider stochastic dynamics with discrete time step $t$ and states $x_tinmathbb{R}^d$ that evolve as



      $$
      mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+f(x_t), Sigma).
      $$



      Moreover, we assume the existence of a unique stationary distribution $rho(x)$ that satisfies



      $$
      intmathrm{p}(x_t|x_{t-1})rho(x_{t-1})mathrm{d}x_{t-1}=rho(x_t).
      $$



      Can $f$ be written as a gradient descent on $lnrho$ transformed by a positive semi-definite matrix $R(x)$,



      $$
      mathrm{p}(x_{t+1}|x_t)=mathcal{N}(x_t+R(x_t)nabla_{x_t}lnrho(x_t),Sigma),?
      $$



      Do I additionally need to assume ergodicity or detailed balance for this to work?










      probability stochastic-processes dynamical-systems markov-chains non-linear-dynamics




      probability stochastic-processes dynamical-systems markov-chains non-linear-dynamics






      share|cite|improve this question















      share|cite|improve this question













      share|cite|improve this question




      share|cite|improve this question








      edited Nov 23 at 18:02

























      asked Nov 20 at 14:47









      danijar

      220313




      220313






















          1 Answer
          1






          active

          oldest

          votes


















          1














          I don't think so. Counterexample:



          State space = ${0,1,2}$



          $p_1: x_{t+1}=x_t+1 $ mod $ 3$



          $p_2: x_{t+1}=x_t-1 $ mod $ 3$



          so $Sigma=0$.



          Then the same $rho(x_t)$ belongs to both $p_1$ and $p_2$.






          share|cite|improve this answer























          • Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
            – danijar
            Nov 22 at 20:12












          • They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
            – Ian
            Nov 22 at 23:06












          • @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
            – Smind
            Nov 23 at 11:17












          • Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
            – danijar
            Nov 23 at 15:18











          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "69"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          noCode: true, onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3006396%2fdescribe-discrete-time-langevin-dynamics-by-its-stationary-distribution%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          1














          I don't think so. Counterexample:



          State space = ${0,1,2}$



          $p_1: x_{t+1}=x_t+1 $ mod $ 3$



          $p_2: x_{t+1}=x_t-1 $ mod $ 3$



          so $Sigma=0$.



          Then the same $rho(x_t)$ belongs to both $p_1$ and $p_2$.






          share|cite|improve this answer























          • Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
            – danijar
            Nov 22 at 20:12












          • They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
            – Ian
            Nov 22 at 23:06












          • @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
            – Smind
            Nov 23 at 11:17












          • Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
            – danijar
            Nov 23 at 15:18
















          1














          I don't think so. Counterexample:



          State space = ${0,1,2}$



          $p_1: x_{t+1}=x_t+1 $ mod $ 3$



          $p_2: x_{t+1}=x_t-1 $ mod $ 3$



          so $Sigma=0$.



          Then the same $rho(x_t)$ belongs to both $p_1$ and $p_2$.






          share|cite|improve this answer























          • Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
            – danijar
            Nov 22 at 20:12












          • They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
            – Ian
            Nov 22 at 23:06












          • @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
            – Smind
            Nov 23 at 11:17












          • Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
            – danijar
            Nov 23 at 15:18














          1












          1








          1






          I don't think so. Counterexample:



          State space = ${0,1,2}$



          $p_1: x_{t+1}=x_t+1 $ mod $ 3$



          $p_2: x_{t+1}=x_t-1 $ mod $ 3$



          so $Sigma=0$.



          Then the same $rho(x_t)$ belongs to both $p_1$ and $p_2$.






          share|cite|improve this answer














          I don't think so. Counterexample:



          State space = ${0,1,2}$



          $p_1: x_{t+1}=x_t+1 $ mod $ 3$



          $p_2: x_{t+1}=x_t-1 $ mod $ 3$



          so $Sigma=0$.



          Then the same $rho(x_t)$ belongs to both $p_1$ and $p_2$.







          share|cite|improve this answer














          share|cite|improve this answer



          share|cite|improve this answer








          edited Nov 23 at 17:45

























          answered Nov 22 at 13:49









          Smind

          163




          163












          • Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
            – danijar
            Nov 22 at 20:12












          • They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
            – Ian
            Nov 22 at 23:06












          • @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
            – Smind
            Nov 23 at 11:17












          • Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
            – danijar
            Nov 23 at 15:18


















          • Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
            – danijar
            Nov 22 at 20:12












          • They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
            – Ian
            Nov 22 at 23:06












          • @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
            – Smind
            Nov 23 at 11:17












          • Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
            – danijar
            Nov 23 at 15:18
















          Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
          – danijar
          Nov 22 at 20:12






          Thanks! Do you mean the deterministic systems $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t+1 ,text{mod}, 3)$ and $p_1(x_{t+1}|x_t)=delta(x_{t+1}=x_t-1 ,text{mod}, 3)$? I think these systems have no stationary distribution $rho(x)$. Also, technically $x_tinmathbb{R}^d$, but an example with discrete states could already be helpful.
          – danijar
          Nov 22 at 20:12














          They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
          – Ian
          Nov 22 at 23:06






          They do have a stationary distribution, it's uniform. It isn't ergodic, but even with ergodicity you cannot do this. The canonical way to do it is called reversible dynamics.
          – Ian
          Nov 22 at 23:06














          @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
          – Smind
          Nov 23 at 11:17






          @danijar thanks, that's right. I agree with Ian that the stationary distribution is uniform. We can generalize this example to continuous $x_t$ by making $f_{1},f_2$ that go clockwise resp. counter clockwise around the same point. They'll have the same stationary distribution.
          – Smind
          Nov 23 at 11:17














          Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
          – danijar
          Nov 23 at 15:18




          Thank you. It makes sense that the stationary distribution is uniform, although it is not ergodic. @Ian, would you mind explaining how this can be done with reversible dynamics? What I had in mind was a discrete-time version of the equilibrium solution to Fokker-Planck in ergodic SDEs with Gaussians noise.
          – danijar
          Nov 23 at 15:18


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Mathematics Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3006396%2fdescribe-discrete-time-langevin-dynamics-by-its-stationary-distribution%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          How to change which sound is reproduced for terminal bell?

          Image
          3 Recently upgrade to 18.10 from 18.04, and using the terminal is becoming annoying: the new alert sound is a drip ( /usr/share/sounds/gnome/default/alerts/drip.ogg ), and the previous one is still located at ( /usr/share/sounds/ubuntu/stereo/bell.ogg ). My issue is that at the gnome-terminal, when autocompleting or pressing left BOTH sounds are reproduced, at the same time. I want to just have ubuntu/stereo/bell.ogg active, not both . At the moment I have been only able to reduce the volume of the alerts (but still both are played), or turn the alerts down (but I dislike both "solutions"). gnome-terminal 18.10 teminal-bell share | improve this question ...
          Read more

          Can I use Tabulator js library in my java Spring + Thymeleaf project?

          Image
          0 I want to add Tabulator js to my java project. I added tabulator.min.js, tabulator.min.css to resources folder. After that added to my html page these dependencies as recommending in readme.md. <link rel="stylesheet" th:href="@{/css/tabulator.min.css}"> <script type="text/javascript" th:src="@{/js/tabulator.min.js}"></script> Now my tables view all elements, that I have in the html page, but not correct. In the documentation of Tabulator is example how to create header and table, var table = new Tabulator("#example-table", { columns:[ {title:"Name", field:"name", sorter:"string", width:200, editor:true}, {title:"Age", field:"age", sorter:"...
          Read more

          Title Spacing in Bjornstrup Chapter, Removing Chapter Number From Contents

          Image
          up vote 5 down vote favorite 2 I would like to make it so that the vertical space below the title of each chapter is the same when using the Bjornstrup package (i.e. so that the descenders do not affect the spacing - as shown by the red arrow in the picture below). I think I'm fairly happy with the spacing around "A chapter", so I'd really like titles without descenders (e.g. "Contents") to have the same vertical spacing as "A chapter" below the title. I would also like to remove the large "0" in the contents title. Any help appreciated, as always. documentclass[11pt,a4paper]{book} % Packages[![enter image description here][1]][1] usepackage[left=3.5cm, right=3.5cm, top=4.3cm, bottom=4.25cm]{geometry} usepackage[utf8]{inputenc} usepackage{libertine} usepackage[libertine]{newtx...
          Read more