Measure that takes samples that is minimized in expectation for a uniformly-distributed random variable?












2














I am having trouble thinking of a function that operates on a set of samples, that is, single-valued random variables between zero and one, $x_i in (0,1), iin{1,2,...I}$, and provides a measure of "non uniformity" of the samples:



$$
F({x_i}) : [0,1]^I to R
$$
This measure would be uniquely minimized (in expectation) when the samples are from a uniform distribution.



My first reaction was to bin these samples by cutting $[0,1]$ into equal-sized peices. Then the standard deviation of the counts across the bins would be my measure, minimized when the counts in each bin are equal.



But the binning allows for "cheating", that is, the samples may not actually be uniformly distributed, but only appear that way due to the binning procedure; a different choice of bins would show the problem.



Any better ideas?



The motivation is to evaluate the quality of a CDF fitting procedure. The fitting procedure is optimal when the CDFs (different CDF for each sample) map all the "real" samples $y_i in R$ uniformly to the [0,1] interval.






sampling descriptive-statistics uniform minimum uniformity







share|cite|improve this question





























    2














    I am having trouble thinking of a function that operates on a set of samples, that is, single-valued random variables between zero and one, $x_i in (0,1), iin{1,2,...I}$, and provides a measure of "non uniformity" of the samples:



    $$
    F({x_i}) : [0,1]^I to R
    $$
    This measure would be uniquely minimized (in expectation) when the samples are from a uniform distribution.



    My first reaction was to bin these samples by cutting $[0,1]$ into equal-sized peices. Then the standard deviation of the counts across the bins would be my measure, minimized when the counts in each bin are equal.



    But the binning allows for "cheating", that is, the samples may not actually be uniformly distributed, but only appear that way due to the binning procedure; a different choice of bins would show the problem.



    Any better ideas?



    The motivation is to evaluate the quality of a CDF fitting procedure. The fitting procedure is optimal when the CDFs (different CDF for each sample) map all the "real" samples $y_i in R$ uniformly to the [0,1] interval.






    sampling descriptive-statistics uniform minimum uniformity







    share|cite|improve this question



























      2












      2








      2







      I am having trouble thinking of a function that operates on a set of samples, that is, single-valued random variables between zero and one, $x_i in (0,1), iin{1,2,...I}$, and provides a measure of "non uniformity" of the samples:



      $$
      F({x_i}) : [0,1]^I to R
      $$
      This measure would be uniquely minimized (in expectation) when the samples are from a uniform distribution.



      My first reaction was to bin these samples by cutting $[0,1]$ into equal-sized peices. Then the standard deviation of the counts across the bins would be my measure, minimized when the counts in each bin are equal.



      But the binning allows for "cheating", that is, the samples may not actually be uniformly distributed, but only appear that way due to the binning procedure; a different choice of bins would show the problem.



      Any better ideas?



      The motivation is to evaluate the quality of a CDF fitting procedure. The fitting procedure is optimal when the CDFs (different CDF for each sample) map all the "real" samples $y_i in R$ uniformly to the [0,1] interval.






      sampling descriptive-statistics uniform minimum uniformity







      share|cite|improve this question















      I am having trouble thinking of a function that operates on a set of samples, that is, single-valued random variables between zero and one, $x_i in (0,1), iin{1,2,...I}$, and provides a measure of "non uniformity" of the samples:



      $$
      F({x_i}) : [0,1]^I to R
      $$
      This measure would be uniquely minimized (in expectation) when the samples are from a uniform distribution.



      My first reaction was to bin these samples by cutting $[0,1]$ into equal-sized peices. Then the standard deviation of the counts across the bins would be my measure, minimized when the counts in each bin are equal.



      But the binning allows for "cheating", that is, the samples may not actually be uniformly distributed, but only appear that way due to the binning procedure; a different choice of bins would show the problem.



      Any better ideas?



      The motivation is to evaluate the quality of a CDF fitting procedure. The fitting procedure is optimal when the CDFs (different CDF for each sample) map all the "real" samples $y_i in R$ uniformly to the [0,1] interval.






      sampling descriptive-statistics uniform minimum uniformity




      sampling descriptive-statistics uniform minimum uniformity






      share|cite|improve this question















      share|cite|improve this question













      share|cite|improve this question




      share|cite|improve this question








      edited Dec 9 at 6:18

























      asked Dec 8 at 5:00









      OrangeSherbet

      1155




      1155






















          2 Answers
          2






          active

          oldest

          votes


















          5














          Like @Chris said, you can use some kind of divergence or information theory ideas. KL divergence like he said is good, or more generally f-divergence, etc., but more simply just use entropy. You said minimized when uniform, so entropy would be maximized, so do negative entropy. You said maps to real numbers, maybe you want for easier comparison, [0,1] interval, so use (negative) normalized entropy (divide by logN):



          https://math.stackexchange.com/questions/395121/how-entropy-scales-with-sample-size



          Or since you said the end goal is to compare some empirical CDF's, then depending on your problem maybe there is a more direct way you can do this like:



          KS test:
          https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test



          Anderson-Darling:
          https://en.wikipedia.org/wiki/Anderson%E2%80%93Darling_test



          Cramer von Mises
          https://en.wikipedia.org/wiki/Cram%C3%A9r%E2%80%93von_Mises_criterion



          etc.






          share|cite|improve this answer

















          • 2




            The biggest revelation for me was realizing I could calculate the "empirical CDF" using the samples, and then use distance measures between the empirical CDF and the cdf for the uniform distribution (a straight line from [0,0] to [1,1]), all without binning anything.
            – OrangeSherbet
            Dec 8 at 10:52





















          2














          Take your samples and form a random variable $X$. To measure the non-uniformity of the samples measure the KL divergence between $X$ and $X'$ as $KL(X||X')$.






          share|cite|improve this answer





















            Your Answer





            StackExchange.ifUsing("editor", function () {
            return StackExchange.using("mathjaxEditing", function () {
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            });
            });
            }, "mathjax-editing");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "65"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f380927%2fmeasure-that-takes-samples-that-is-minimized-in-expectation-for-a-uniformly-dist%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            5














            Like @Chris said, you can use some kind of divergence or information theory ideas. KL divergence like he said is good, or more generally f-divergence, etc., but more simply just use entropy. You said minimized when uniform, so entropy would be maximized, so do negative entropy. You said maps to real numbers, maybe you want for easier comparison, [0,1] interval, so use (negative) normalized entropy (divide by logN):



            https://math.stackexchange.com/questions/395121/how-entropy-scales-with-sample-size



            Or since you said the end goal is to compare some empirical CDF's, then depending on your problem maybe there is a more direct way you can do this like:



            KS test:
            https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test



            Anderson-Darling:
            https://en.wikipedia.org/wiki/Anderson%E2%80%93Darling_test



            Cramer von Mises
            https://en.wikipedia.org/wiki/Cram%C3%A9r%E2%80%93von_Mises_criterion



            etc.






            share|cite|improve this answer

















            • 2




              The biggest revelation for me was realizing I could calculate the "empirical CDF" using the samples, and then use distance measures between the empirical CDF and the cdf for the uniform distribution (a straight line from [0,0] to [1,1]), all without binning anything.
              – OrangeSherbet
              Dec 8 at 10:52


















            5














            Like @Chris said, you can use some kind of divergence or information theory ideas. KL divergence like he said is good, or more generally f-divergence, etc., but more simply just use entropy. You said minimized when uniform, so entropy would be maximized, so do negative entropy. You said maps to real numbers, maybe you want for easier comparison, [0,1] interval, so use (negative) normalized entropy (divide by logN):



            https://math.stackexchange.com/questions/395121/how-entropy-scales-with-sample-size



            Or since you said the end goal is to compare some empirical CDF's, then depending on your problem maybe there is a more direct way you can do this like:



            KS test:
            https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test



            Anderson-Darling:
            https://en.wikipedia.org/wiki/Anderson%E2%80%93Darling_test



            Cramer von Mises
            https://en.wikipedia.org/wiki/Cram%C3%A9r%E2%80%93von_Mises_criterion



            etc.






            share|cite|improve this answer

















            • 2




              The biggest revelation for me was realizing I could calculate the "empirical CDF" using the samples, and then use distance measures between the empirical CDF and the cdf for the uniform distribution (a straight line from [0,0] to [1,1]), all without binning anything.
              – OrangeSherbet
              Dec 8 at 10:52
















            5












            5








            5






            Like @Chris said, you can use some kind of divergence or information theory ideas. KL divergence like he said is good, or more generally f-divergence, etc., but more simply just use entropy. You said minimized when uniform, so entropy would be maximized, so do negative entropy. You said maps to real numbers, maybe you want for easier comparison, [0,1] interval, so use (negative) normalized entropy (divide by logN):



            https://math.stackexchange.com/questions/395121/how-entropy-scales-with-sample-size



            Or since you said the end goal is to compare some empirical CDF's, then depending on your problem maybe there is a more direct way you can do this like:



            KS test:
            https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test



            Anderson-Darling:
            https://en.wikipedia.org/wiki/Anderson%E2%80%93Darling_test



            Cramer von Mises
            https://en.wikipedia.org/wiki/Cram%C3%A9r%E2%80%93von_Mises_criterion



            etc.






            share|cite|improve this answer












            Like @Chris said, you can use some kind of divergence or information theory ideas. KL divergence like he said is good, or more generally f-divergence, etc., but more simply just use entropy. You said minimized when uniform, so entropy would be maximized, so do negative entropy. You said maps to real numbers, maybe you want for easier comparison, [0,1] interval, so use (negative) normalized entropy (divide by logN):



            https://math.stackexchange.com/questions/395121/how-entropy-scales-with-sample-size



            Or since you said the end goal is to compare some empirical CDF's, then depending on your problem maybe there is a more direct way you can do this like:



            KS test:
            https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test



            Anderson-Darling:
            https://en.wikipedia.org/wiki/Anderson%E2%80%93Darling_test



            Cramer von Mises
            https://en.wikipedia.org/wiki/Cram%C3%A9r%E2%80%93von_Mises_criterion



            etc.







            share|cite|improve this answer












            share|cite|improve this answer



            share|cite|improve this answer










            answered Dec 8 at 6:35









            123learn

            761




            761








            • 2




              The biggest revelation for me was realizing I could calculate the "empirical CDF" using the samples, and then use distance measures between the empirical CDF and the cdf for the uniform distribution (a straight line from [0,0] to [1,1]), all without binning anything.
              – OrangeSherbet
              Dec 8 at 10:52
















            • 2




              The biggest revelation for me was realizing I could calculate the "empirical CDF" using the samples, and then use distance measures between the empirical CDF and the cdf for the uniform distribution (a straight line from [0,0] to [1,1]), all without binning anything.
              – OrangeSherbet
              Dec 8 at 10:52










            2




            2




            The biggest revelation for me was realizing I could calculate the "empirical CDF" using the samples, and then use distance measures between the empirical CDF and the cdf for the uniform distribution (a straight line from [0,0] to [1,1]), all without binning anything.
            – OrangeSherbet
            Dec 8 at 10:52






            The biggest revelation for me was realizing I could calculate the "empirical CDF" using the samples, and then use distance measures between the empirical CDF and the cdf for the uniform distribution (a straight line from [0,0] to [1,1]), all without binning anything.
            – OrangeSherbet
            Dec 8 at 10:52















            2














            Take your samples and form a random variable $X$. To measure the non-uniformity of the samples measure the KL divergence between $X$ and $X'$ as $KL(X||X')$.






            share|cite|improve this answer


























              2














              Take your samples and form a random variable $X$. To measure the non-uniformity of the samples measure the KL divergence between $X$ and $X'$ as $KL(X||X')$.






              share|cite|improve this answer
























                2












                2








                2






                Take your samples and form a random variable $X$. To measure the non-uniformity of the samples measure the KL divergence between $X$ and $X'$ as $KL(X||X')$.






                share|cite|improve this answer












                Take your samples and form a random variable $X$. To measure the non-uniformity of the samples measure the KL divergence between $X$ and $X'$ as $KL(X||X')$.







                share|cite|improve this answer












                share|cite|improve this answer



                share|cite|improve this answer










                answered Dec 8 at 6:00









                Chris

                514211




                514211






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Cross Validated!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    Use MathJax to format equations. MathJax reference.


                    To learn more, see our tips on writing great answers.





                    Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                    Please pay close attention to the following guidance:


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f380927%2fmeasure-that-takes-samples-that-is-minimized-in-expectation-for-a-uniformly-dist%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    How to change which sound is reproduced for terminal bell?

                    Image
                    3 Recently upgrade to 18.10 from 18.04, and using the terminal is becoming annoying: the new alert sound is a drip ( /usr/share/sounds/gnome/default/alerts/drip.ogg ), and the previous one is still located at ( /usr/share/sounds/ubuntu/stereo/bell.ogg ). My issue is that at the gnome-terminal, when autocompleting or pressing left BOTH sounds are reproduced, at the same time. I want to just have ubuntu/stereo/bell.ogg active, not both . At the moment I have been only able to reduce the volume of the alerts (but still both are played), or turn the alerts down (but I dislike both "solutions"). gnome-terminal 18.10 teminal-bell share | improve this question ...
                    Read more

                    Can I use Tabulator js library in my java Spring + Thymeleaf project?

                    Image
                    0 I want to add Tabulator js to my java project. I added tabulator.min.js, tabulator.min.css to resources folder. After that added to my html page these dependencies as recommending in readme.md. <link rel="stylesheet" th:href="@{/css/tabulator.min.css}"> <script type="text/javascript" th:src="@{/js/tabulator.min.js}"></script> Now my tables view all elements, that I have in the html page, but not correct. In the documentation of Tabulator is example how to create header and table, var table = new Tabulator("#example-table", { columns:[ {title:"Name", field:"name", sorter:"string", width:200, editor:true}, {title:"Age", field:"age", sorter:"...
                    Read more

                    Title Spacing in Bjornstrup Chapter, Removing Chapter Number From Contents

                    Image
                    up vote 5 down vote favorite 2 I would like to make it so that the vertical space below the title of each chapter is the same when using the Bjornstrup package (i.e. so that the descenders do not affect the spacing - as shown by the red arrow in the picture below). I think I'm fairly happy with the spacing around "A chapter", so I'd really like titles without descenders (e.g. "Contents") to have the same vertical spacing as "A chapter" below the title. I would also like to remove the large "0" in the contents title. Any help appreciated, as always. documentclass[11pt,a4paper]{book} % Packages[![enter image description here][1]][1] usepackage[left=3.5cm, right=3.5cm, top=4.3cm, bottom=4.25cm]{geometry} usepackage[utf8]{inputenc} usepackage{libertine} usepackage[libertine]{newtx...
                    Read more