Classification of states and chains in Markov Chain












0












$begingroup$


Let $S={{1,2,3,4,5}}$. Find out which states are: persistent, transient, null, non-null, periodic, aperiodic, ergodic and absorbing. Find closed and irreducible sets of a states. For closed sets that are irreducible find mean recurrence times and stationary distribution, when



$
P=
begin{bmatrix}
1 & 0 & 0 & 0 & 0 \
1/4 & 1/4 & 1/8 & 1/4 & 1/8 \
0 & 0 & 1/4 & 1/4 & 1/2 \
0 & 0 & 1/2 & 1/2 & 0 \
0 & 0 & 1/4 & 3/4 & 0
end{bmatrix}
$




Closed sets: ${{3,4}}, {{3,5}}$



Irreducible sets: ${{3,4}}, {{3,5}}$



Sets ${{3,4}}, {{3,5}}$ consists of states: persistent and non-null



Transient set: ${{2}}$



I'm not sure how to find out if a state is periodic or aperiodic. If $p_{ii}>0$, then the period equals $1$, and so all the states are periodic?



Please check the above, correct me if I'm wrong anywhere, and help me with the other part of the exercise. Any will be much appreciated.






probability-theory stochastic-processes markov-chains







share|cite|improve this question









$endgroup$

















    0












    $begingroup$


    Let $S={{1,2,3,4,5}}$. Find out which states are: persistent, transient, null, non-null, periodic, aperiodic, ergodic and absorbing. Find closed and irreducible sets of a states. For closed sets that are irreducible find mean recurrence times and stationary distribution, when



    $
    P=
    begin{bmatrix}
    1 & 0 & 0 & 0 & 0 \
    1/4 & 1/4 & 1/8 & 1/4 & 1/8 \
    0 & 0 & 1/4 & 1/4 & 1/2 \
    0 & 0 & 1/2 & 1/2 & 0 \
    0 & 0 & 1/4 & 3/4 & 0
    end{bmatrix}
    $




    Closed sets: ${{3,4}}, {{3,5}}$



    Irreducible sets: ${{3,4}}, {{3,5}}$



    Sets ${{3,4}}, {{3,5}}$ consists of states: persistent and non-null



    Transient set: ${{2}}$



    I'm not sure how to find out if a state is periodic or aperiodic. If $p_{ii}>0$, then the period equals $1$, and so all the states are periodic?



    Please check the above, correct me if I'm wrong anywhere, and help me with the other part of the exercise. Any will be much appreciated.






    probability-theory stochastic-processes markov-chains







    share|cite|improve this question









    $endgroup$















      0












      0








      0





      $begingroup$


      Let $S={{1,2,3,4,5}}$. Find out which states are: persistent, transient, null, non-null, periodic, aperiodic, ergodic and absorbing. Find closed and irreducible sets of a states. For closed sets that are irreducible find mean recurrence times and stationary distribution, when



      $
      P=
      begin{bmatrix}
      1 & 0 & 0 & 0 & 0 \
      1/4 & 1/4 & 1/8 & 1/4 & 1/8 \
      0 & 0 & 1/4 & 1/4 & 1/2 \
      0 & 0 & 1/2 & 1/2 & 0 \
      0 & 0 & 1/4 & 3/4 & 0
      end{bmatrix}
      $




      Closed sets: ${{3,4}}, {{3,5}}$



      Irreducible sets: ${{3,4}}, {{3,5}}$



      Sets ${{3,4}}, {{3,5}}$ consists of states: persistent and non-null



      Transient set: ${{2}}$



      I'm not sure how to find out if a state is periodic or aperiodic. If $p_{ii}>0$, then the period equals $1$, and so all the states are periodic?



      Please check the above, correct me if I'm wrong anywhere, and help me with the other part of the exercise. Any will be much appreciated.






      probability-theory stochastic-processes markov-chains







      share|cite|improve this question









      $endgroup$




      Let $S={{1,2,3,4,5}}$. Find out which states are: persistent, transient, null, non-null, periodic, aperiodic, ergodic and absorbing. Find closed and irreducible sets of a states. For closed sets that are irreducible find mean recurrence times and stationary distribution, when



      $
      P=
      begin{bmatrix}
      1 & 0 & 0 & 0 & 0 \
      1/4 & 1/4 & 1/8 & 1/4 & 1/8 \
      0 & 0 & 1/4 & 1/4 & 1/2 \
      0 & 0 & 1/2 & 1/2 & 0 \
      0 & 0 & 1/4 & 3/4 & 0
      end{bmatrix}
      $




      Closed sets: ${{3,4}}, {{3,5}}$



      Irreducible sets: ${{3,4}}, {{3,5}}$



      Sets ${{3,4}}, {{3,5}}$ consists of states: persistent and non-null



      Transient set: ${{2}}$



      I'm not sure how to find out if a state is periodic or aperiodic. If $p_{ii}>0$, then the period equals $1$, and so all the states are periodic?



      Please check the above, correct me if I'm wrong anywhere, and help me with the other part of the exercise. Any will be much appreciated.






      probability-theory stochastic-processes markov-chains




      probability-theory stochastic-processes markov-chains






      share|cite|improve this question













      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked Dec 15 '18 at 9:16









      MacAbraMacAbra

      269210




      269210






















          1 Answer
          1






          active

          oldest

          votes


















          2












          $begingroup$

          ${1}$ and ${3,4,5}$ are closed sets, $2$ is transient and all recurrent states are aperiodic. [ ${3,4}$ is not a closed set because you can go from $3$ to $5$]. Of course, $1$ is an absorbing state.






          share|cite|improve this answer









          $endgroup$









          • 1




            $begingroup$
            Thank you for correcting my closed sets. But what about irreducible sets? I think these two are also irreducible, can I find any other ones? Can you explain me a little more why all recurrent states are aperiodic, and generally how do I find if a state is periodic or not?
            $endgroup$
            – MacAbra
            Dec 15 '18 at 15:03






          • 1




            $begingroup$
            From $3$ or $4$ you cannot go to $5$ so ${3,4,5}$ is not irreducible. Period of state $i$ is the gcd of ${n:p_{ii}^{(n)} >0$. If $p_{ii} >0$ the the state is aperiodoc (i.e. has period $1$) because the gcd of any set of integers containing $1$ is $1$.
            $endgroup$
            – Kavi Rama Murthy
            Dec 15 '18 at 23:44














          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "69"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          noCode: true, onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3040307%2fclassification-of-states-and-chains-in-markov-chain%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          2












          $begingroup$

          ${1}$ and ${3,4,5}$ are closed sets, $2$ is transient and all recurrent states are aperiodic. [ ${3,4}$ is not a closed set because you can go from $3$ to $5$]. Of course, $1$ is an absorbing state.






          share|cite|improve this answer









          $endgroup$









          • 1




            $begingroup$
            Thank you for correcting my closed sets. But what about irreducible sets? I think these two are also irreducible, can I find any other ones? Can you explain me a little more why all recurrent states are aperiodic, and generally how do I find if a state is periodic or not?
            $endgroup$
            – MacAbra
            Dec 15 '18 at 15:03






          • 1




            $begingroup$
            From $3$ or $4$ you cannot go to $5$ so ${3,4,5}$ is not irreducible. Period of state $i$ is the gcd of ${n:p_{ii}^{(n)} >0$. If $p_{ii} >0$ the the state is aperiodoc (i.e. has period $1$) because the gcd of any set of integers containing $1$ is $1$.
            $endgroup$
            – Kavi Rama Murthy
            Dec 15 '18 at 23:44


















          2












          $begingroup$

          ${1}$ and ${3,4,5}$ are closed sets, $2$ is transient and all recurrent states are aperiodic. [ ${3,4}$ is not a closed set because you can go from $3$ to $5$]. Of course, $1$ is an absorbing state.






          share|cite|improve this answer









          $endgroup$









          • 1




            $begingroup$
            Thank you for correcting my closed sets. But what about irreducible sets? I think these two are also irreducible, can I find any other ones? Can you explain me a little more why all recurrent states are aperiodic, and generally how do I find if a state is periodic or not?
            $endgroup$
            – MacAbra
            Dec 15 '18 at 15:03






          • 1




            $begingroup$
            From $3$ or $4$ you cannot go to $5$ so ${3,4,5}$ is not irreducible. Period of state $i$ is the gcd of ${n:p_{ii}^{(n)} >0$. If $p_{ii} >0$ the the state is aperiodoc (i.e. has period $1$) because the gcd of any set of integers containing $1$ is $1$.
            $endgroup$
            – Kavi Rama Murthy
            Dec 15 '18 at 23:44
















          2












          2








          2





          $begingroup$

          ${1}$ and ${3,4,5}$ are closed sets, $2$ is transient and all recurrent states are aperiodic. [ ${3,4}$ is not a closed set because you can go from $3$ to $5$]. Of course, $1$ is an absorbing state.






          share|cite|improve this answer









          $endgroup$



          ${1}$ and ${3,4,5}$ are closed sets, $2$ is transient and all recurrent states are aperiodic. [ ${3,4}$ is not a closed set because you can go from $3$ to $5$]. Of course, $1$ is an absorbing state.







          share|cite|improve this answer












          share|cite|improve this answer



          share|cite|improve this answer










          answered Dec 15 '18 at 12:22









          Kavi Rama MurthyKavi Rama Murthy

          74.1k53270




          74.1k53270








          • 1




            $begingroup$
            Thank you for correcting my closed sets. But what about irreducible sets? I think these two are also irreducible, can I find any other ones? Can you explain me a little more why all recurrent states are aperiodic, and generally how do I find if a state is periodic or not?
            $endgroup$
            – MacAbra
            Dec 15 '18 at 15:03






          • 1




            $begingroup$
            From $3$ or $4$ you cannot go to $5$ so ${3,4,5}$ is not irreducible. Period of state $i$ is the gcd of ${n:p_{ii}^{(n)} >0$. If $p_{ii} >0$ the the state is aperiodoc (i.e. has period $1$) because the gcd of any set of integers containing $1$ is $1$.
            $endgroup$
            – Kavi Rama Murthy
            Dec 15 '18 at 23:44
















          • 1




            $begingroup$
            Thank you for correcting my closed sets. But what about irreducible sets? I think these two are also irreducible, can I find any other ones? Can you explain me a little more why all recurrent states are aperiodic, and generally how do I find if a state is periodic or not?
            $endgroup$
            – MacAbra
            Dec 15 '18 at 15:03






          • 1




            $begingroup$
            From $3$ or $4$ you cannot go to $5$ so ${3,4,5}$ is not irreducible. Period of state $i$ is the gcd of ${n:p_{ii}^{(n)} >0$. If $p_{ii} >0$ the the state is aperiodoc (i.e. has period $1$) because the gcd of any set of integers containing $1$ is $1$.
            $endgroup$
            – Kavi Rama Murthy
            Dec 15 '18 at 23:44










          1




          1




          $begingroup$
          Thank you for correcting my closed sets. But what about irreducible sets? I think these two are also irreducible, can I find any other ones? Can you explain me a little more why all recurrent states are aperiodic, and generally how do I find if a state is periodic or not?
          $endgroup$
          – MacAbra
          Dec 15 '18 at 15:03




          $begingroup$
          Thank you for correcting my closed sets. But what about irreducible sets? I think these two are also irreducible, can I find any other ones? Can you explain me a little more why all recurrent states are aperiodic, and generally how do I find if a state is periodic or not?
          $endgroup$
          – MacAbra
          Dec 15 '18 at 15:03




          1




          1




          $begingroup$
          From $3$ or $4$ you cannot go to $5$ so ${3,4,5}$ is not irreducible. Period of state $i$ is the gcd of ${n:p_{ii}^{(n)} >0$. If $p_{ii} >0$ the the state is aperiodoc (i.e. has period $1$) because the gcd of any set of integers containing $1$ is $1$.
          $endgroup$
          – Kavi Rama Murthy
          Dec 15 '18 at 23:44






          $begingroup$
          From $3$ or $4$ you cannot go to $5$ so ${3,4,5}$ is not irreducible. Period of state $i$ is the gcd of ${n:p_{ii}^{(n)} >0$. If $p_{ii} >0$ the the state is aperiodoc (i.e. has period $1$) because the gcd of any set of integers containing $1$ is $1$.
          $endgroup$
          – Kavi Rama Murthy
          Dec 15 '18 at 23:44




















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Mathematics Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3040307%2fclassification-of-states-and-chains-in-markov-chain%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          mysqli_query(): Empty query in /home/lucindabrummitt/public_html/blog/wp-includes/wp-db.php on line 1924

          Image
          0 I just uploaded a new plugin to my wordpress site and I got this warning: mysqli_query(): Empty query in /home/lucindabrummitt/public_html/blog/wp-includes/wp-db.php on line 1924 Usually if I have a problem, I can find answers on google but this one seems pretty particular and I'm not finding any info on it except for this forum. I'm not terribly techie but if someone can tell me how to read this and how to change it I can follow directions pretty well. Thanks All! PS If I don't have the proper tags here could someone add the appropriate tags? php mysql wordpress share | improve this question asked Nov 16 '18 at 20:41 ...
          Read more

          How to change which sound is reproduced for terminal bell?

          Image
          3 Recently upgrade to 18.10 from 18.04, and using the terminal is becoming annoying: the new alert sound is a drip ( /usr/share/sounds/gnome/default/alerts/drip.ogg ), and the previous one is still located at ( /usr/share/sounds/ubuntu/stereo/bell.ogg ). My issue is that at the gnome-terminal, when autocompleting or pressing left BOTH sounds are reproduced, at the same time. I want to just have ubuntu/stereo/bell.ogg active, not both . At the moment I have been only able to reduce the volume of the alerts (but still both are played), or turn the alerts down (but I dislike both "solutions"). gnome-terminal 18.10 teminal-bell share | improve this question ...
          Read more

          Can I use Tabulator js library in my java Spring + Thymeleaf project?

          Image
          0 I want to add Tabulator js to my java project. I added tabulator.min.js, tabulator.min.css to resources folder. After that added to my html page these dependencies as recommending in readme.md. <link rel="stylesheet" th:href="@{/css/tabulator.min.css}"> <script type="text/javascript" th:src="@{/js/tabulator.min.js}"></script> Now my tables view all elements, that I have in the html page, but not correct. In the documentation of Tabulator is example how to create header and table, var table = new Tabulator("#example-table", { columns:[ {title:"Name", field:"name", sorter:"string", width:200, editor:true}, {title:"Age", field:"age", sorter:"...
          Read more