What are the consequences of removing a single byte from a sha256 hash?











up vote
14
down vote

favorite
3












I'm working on a system (Ethereum) where it is significantly cheaper to store 32 bytes than 33 bytes. I'd like to create a table where data is stored based on its hash.



Sha256 would meet this criteria since it outputs 32 bytes.



However, I'd also like to include a "version" byte in case I need to change the hash algorithm in the future. This would require 33 bytes.



One simple solution is to simply chop off the last byte and only use the first 31 bytes for the lookup.




  1. Does this bias the hash in any way?

  2. My assumption is this would decrease the computational power needed to reverse the hash by 1/256th. Is that correct?

  3. My assumption is this would increase the likelyhood of a hash collision by 25600%. Is that correct?










share|improve this question




























    up vote
    14
    down vote

    favorite
    3












    I'm working on a system (Ethereum) where it is significantly cheaper to store 32 bytes than 33 bytes. I'd like to create a table where data is stored based on its hash.



    Sha256 would meet this criteria since it outputs 32 bytes.



    However, I'd also like to include a "version" byte in case I need to change the hash algorithm in the future. This would require 33 bytes.



    One simple solution is to simply chop off the last byte and only use the first 31 bytes for the lookup.




    1. Does this bias the hash in any way?

    2. My assumption is this would decrease the computational power needed to reverse the hash by 1/256th. Is that correct?

    3. My assumption is this would increase the likelyhood of a hash collision by 25600%. Is that correct?










    share|improve this question


























      up vote
      14
      down vote

      favorite
      3









      up vote
      14
      down vote

      favorite
      3






      3





      I'm working on a system (Ethereum) where it is significantly cheaper to store 32 bytes than 33 bytes. I'd like to create a table where data is stored based on its hash.



      Sha256 would meet this criteria since it outputs 32 bytes.



      However, I'd also like to include a "version" byte in case I need to change the hash algorithm in the future. This would require 33 bytes.



      One simple solution is to simply chop off the last byte and only use the first 31 bytes for the lookup.




      1. Does this bias the hash in any way?

      2. My assumption is this would decrease the computational power needed to reverse the hash by 1/256th. Is that correct?

      3. My assumption is this would increase the likelyhood of a hash collision by 25600%. Is that correct?










      share|improve this question















      I'm working on a system (Ethereum) where it is significantly cheaper to store 32 bytes than 33 bytes. I'd like to create a table where data is stored based on its hash.



      Sha256 would meet this criteria since it outputs 32 bytes.



      However, I'd also like to include a "version" byte in case I need to change the hash algorithm in the future. This would require 33 bytes.



      One simple solution is to simply chop off the last byte and only use the first 31 bytes for the lookup.




      1. Does this bias the hash in any way?

      2. My assumption is this would decrease the computational power needed to reverse the hash by 1/256th. Is that correct?

      3. My assumption is this would increase the likelyhood of a hash collision by 25600%. Is that correct?







      hash collision-resistance sha-256 preimage-resistance






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 25 at 23:03









      kelalaka

      3,51711331




      3,51711331










      asked Nov 25 at 21:36









      Aakil Fernandes

      17515




      17515






















          1 Answer
          1






          active

          oldest

          votes

















          up vote
          18
          down vote



          accepted












          1. Does this bias the hash in any way?




          We want the avalanche criteria on the output bits, that is a change in the any of input bit must randomly affect half of the output bits. Each bit of the hash function must depend on the input bits; removing one bit doesn't affect the others.





          1. My assumption is this would decrease the computational power needed to reverse the hash by 1/256th. Is that correct?




          First of all, hash functions are not really reversible since they are compression functions, that is, they map from a large input space to a shorter space.



          $$ H:{0,1}^* rightarrow {0,1}^l$$



          If we want to talk about collision resistance, see the next answer.
          For generic pre-image search, yes; it will decrease the computational power, as you noted.





          1. My assumption is this would increase the likelihood of a hash collision by 25600%. Is that correct?




          Collision resistance is measured by the generic birthday attack, that is, $sqrt {2^l}$, $l$ being the output size of the hash function. SHA256 has $sqrt {2^{256}} = 2^{128}$ generic birthday attack time.



          In your case we will have $sqrt {2^{256-8}} = 2^{124}$ as generic birthday attack time. Thus, we have a $2^{4}= 16$ speed-up in the attack time.



          TL;DR: Truncating the hash to 31 bytes will be safe (see also this stack exchange answer).





          Note 1: Bitcoin miners reached $approx2^{91.6}$ SHA-256 hashes per year in 25 September 2018.



          Note 2: SHA-224 defined in FIPS180-4 is calculated by truncating the SHA-256 hash value (and using different initial constants, for various technical reasons, so the value is not the same as the first 28 bytes of the SHA-256 value).






          share|improve this answer



















          • 1




            Thanks very much! Quick question, why did you subtract 256 by 32 when calculating the generic birthday attack time? I'm decreasing the output size by 1 byte (8 bits) not 32 bits.
            – Aakil Fernandes
            Nov 25 at 22:04






          • 2




            it should be 8, my mistake. let me correct.
            – kelalaka
            Nov 25 at 22:05








          • 3




            @kelalaka - FWIW, what you are proposing is exactly how some SHA variants work. For example SHA-224 is often implemented by generating SHA-256 and chopping off 4 bytes. So you can simply claim you're using SHA-248
            – slebetman
            Nov 26 at 9:57






          • 8




            @slebetman, IIRC, SHA-224 uses different initialization constants, too, so it doesn't give the same results as SHA-256 chopped off to length. And no, you can't say "you're using SHA-248", since no such thing exists in any standards. You could say you're using SHA-256 truncated to 248 bits, since, well, that's what you are doing.
            – ilkkachu
            Nov 26 at 11:19











          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "281"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          noCode: true, onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcrypto.stackexchange.com%2fquestions%2f64314%2fwhat-are-the-consequences-of-removing-a-single-byte-from-a-sha256-hash%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          18
          down vote



          accepted












          1. Does this bias the hash in any way?




          We want the avalanche criteria on the output bits, that is a change in the any of input bit must randomly affect half of the output bits. Each bit of the hash function must depend on the input bits; removing one bit doesn't affect the others.





          1. My assumption is this would decrease the computational power needed to reverse the hash by 1/256th. Is that correct?




          First of all, hash functions are not really reversible since they are compression functions, that is, they map from a large input space to a shorter space.



          $$ H:{0,1}^* rightarrow {0,1}^l$$



          If we want to talk about collision resistance, see the next answer.
          For generic pre-image search, yes; it will decrease the computational power, as you noted.





          1. My assumption is this would increase the likelihood of a hash collision by 25600%. Is that correct?




          Collision resistance is measured by the generic birthday attack, that is, $sqrt {2^l}$, $l$ being the output size of the hash function. SHA256 has $sqrt {2^{256}} = 2^{128}$ generic birthday attack time.



          In your case we will have $sqrt {2^{256-8}} = 2^{124}$ as generic birthday attack time. Thus, we have a $2^{4}= 16$ speed-up in the attack time.



          TL;DR: Truncating the hash to 31 bytes will be safe (see also this stack exchange answer).





          Note 1: Bitcoin miners reached $approx2^{91.6}$ SHA-256 hashes per year in 25 September 2018.



          Note 2: SHA-224 defined in FIPS180-4 is calculated by truncating the SHA-256 hash value (and using different initial constants, for various technical reasons, so the value is not the same as the first 28 bytes of the SHA-256 value).






          share|improve this answer



















          • 1




            Thanks very much! Quick question, why did you subtract 256 by 32 when calculating the generic birthday attack time? I'm decreasing the output size by 1 byte (8 bits) not 32 bits.
            – Aakil Fernandes
            Nov 25 at 22:04






          • 2




            it should be 8, my mistake. let me correct.
            – kelalaka
            Nov 25 at 22:05








          • 3




            @kelalaka - FWIW, what you are proposing is exactly how some SHA variants work. For example SHA-224 is often implemented by generating SHA-256 and chopping off 4 bytes. So you can simply claim you're using SHA-248
            – slebetman
            Nov 26 at 9:57






          • 8




            @slebetman, IIRC, SHA-224 uses different initialization constants, too, so it doesn't give the same results as SHA-256 chopped off to length. And no, you can't say "you're using SHA-248", since no such thing exists in any standards. You could say you're using SHA-256 truncated to 248 bits, since, well, that's what you are doing.
            – ilkkachu
            Nov 26 at 11:19















          up vote
          18
          down vote



          accepted












          1. Does this bias the hash in any way?




          We want the avalanche criteria on the output bits, that is a change in the any of input bit must randomly affect half of the output bits. Each bit of the hash function must depend on the input bits; removing one bit doesn't affect the others.





          1. My assumption is this would decrease the computational power needed to reverse the hash by 1/256th. Is that correct?




          First of all, hash functions are not really reversible since they are compression functions, that is, they map from a large input space to a shorter space.



          $$ H:{0,1}^* rightarrow {0,1}^l$$



          If we want to talk about collision resistance, see the next answer.
          For generic pre-image search, yes; it will decrease the computational power, as you noted.





          1. My assumption is this would increase the likelihood of a hash collision by 25600%. Is that correct?




          Collision resistance is measured by the generic birthday attack, that is, $sqrt {2^l}$, $l$ being the output size of the hash function. SHA256 has $sqrt {2^{256}} = 2^{128}$ generic birthday attack time.



          In your case we will have $sqrt {2^{256-8}} = 2^{124}$ as generic birthday attack time. Thus, we have a $2^{4}= 16$ speed-up in the attack time.



          TL;DR: Truncating the hash to 31 bytes will be safe (see also this stack exchange answer).





          Note 1: Bitcoin miners reached $approx2^{91.6}$ SHA-256 hashes per year in 25 September 2018.



          Note 2: SHA-224 defined in FIPS180-4 is calculated by truncating the SHA-256 hash value (and using different initial constants, for various technical reasons, so the value is not the same as the first 28 bytes of the SHA-256 value).






          share|improve this answer



















          • 1




            Thanks very much! Quick question, why did you subtract 256 by 32 when calculating the generic birthday attack time? I'm decreasing the output size by 1 byte (8 bits) not 32 bits.
            – Aakil Fernandes
            Nov 25 at 22:04






          • 2




            it should be 8, my mistake. let me correct.
            – kelalaka
            Nov 25 at 22:05








          • 3




            @kelalaka - FWIW, what you are proposing is exactly how some SHA variants work. For example SHA-224 is often implemented by generating SHA-256 and chopping off 4 bytes. So you can simply claim you're using SHA-248
            – slebetman
            Nov 26 at 9:57






          • 8




            @slebetman, IIRC, SHA-224 uses different initialization constants, too, so it doesn't give the same results as SHA-256 chopped off to length. And no, you can't say "you're using SHA-248", since no such thing exists in any standards. You could say you're using SHA-256 truncated to 248 bits, since, well, that's what you are doing.
            – ilkkachu
            Nov 26 at 11:19













          up vote
          18
          down vote



          accepted







          up vote
          18
          down vote



          accepted








          1. Does this bias the hash in any way?




          We want the avalanche criteria on the output bits, that is a change in the any of input bit must randomly affect half of the output bits. Each bit of the hash function must depend on the input bits; removing one bit doesn't affect the others.





          1. My assumption is this would decrease the computational power needed to reverse the hash by 1/256th. Is that correct?




          First of all, hash functions are not really reversible since they are compression functions, that is, they map from a large input space to a shorter space.



          $$ H:{0,1}^* rightarrow {0,1}^l$$



          If we want to talk about collision resistance, see the next answer.
          For generic pre-image search, yes; it will decrease the computational power, as you noted.





          1. My assumption is this would increase the likelihood of a hash collision by 25600%. Is that correct?




          Collision resistance is measured by the generic birthday attack, that is, $sqrt {2^l}$, $l$ being the output size of the hash function. SHA256 has $sqrt {2^{256}} = 2^{128}$ generic birthday attack time.



          In your case we will have $sqrt {2^{256-8}} = 2^{124}$ as generic birthday attack time. Thus, we have a $2^{4}= 16$ speed-up in the attack time.



          TL;DR: Truncating the hash to 31 bytes will be safe (see also this stack exchange answer).





          Note 1: Bitcoin miners reached $approx2^{91.6}$ SHA-256 hashes per year in 25 September 2018.



          Note 2: SHA-224 defined in FIPS180-4 is calculated by truncating the SHA-256 hash value (and using different initial constants, for various technical reasons, so the value is not the same as the first 28 bytes of the SHA-256 value).






          share|improve this answer
















          1. Does this bias the hash in any way?




          We want the avalanche criteria on the output bits, that is a change in the any of input bit must randomly affect half of the output bits. Each bit of the hash function must depend on the input bits; removing one bit doesn't affect the others.





          1. My assumption is this would decrease the computational power needed to reverse the hash by 1/256th. Is that correct?




          First of all, hash functions are not really reversible since they are compression functions, that is, they map from a large input space to a shorter space.



          $$ H:{0,1}^* rightarrow {0,1}^l$$



          If we want to talk about collision resistance, see the next answer.
          For generic pre-image search, yes; it will decrease the computational power, as you noted.





          1. My assumption is this would increase the likelihood of a hash collision by 25600%. Is that correct?




          Collision resistance is measured by the generic birthday attack, that is, $sqrt {2^l}$, $l$ being the output size of the hash function. SHA256 has $sqrt {2^{256}} = 2^{128}$ generic birthday attack time.



          In your case we will have $sqrt {2^{256-8}} = 2^{124}$ as generic birthday attack time. Thus, we have a $2^{4}= 16$ speed-up in the attack time.



          TL;DR: Truncating the hash to 31 bytes will be safe (see also this stack exchange answer).





          Note 1: Bitcoin miners reached $approx2^{91.6}$ SHA-256 hashes per year in 25 September 2018.



          Note 2: SHA-224 defined in FIPS180-4 is calculated by truncating the SHA-256 hash value (and using different initial constants, for various technical reasons, so the value is not the same as the first 28 bytes of the SHA-256 value).







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited yesterday









          dkaeae

          378311




          378311










          answered Nov 25 at 21:57









          kelalaka

          3,51711331




          3,51711331








          • 1




            Thanks very much! Quick question, why did you subtract 256 by 32 when calculating the generic birthday attack time? I'm decreasing the output size by 1 byte (8 bits) not 32 bits.
            – Aakil Fernandes
            Nov 25 at 22:04






          • 2




            it should be 8, my mistake. let me correct.
            – kelalaka
            Nov 25 at 22:05








          • 3




            @kelalaka - FWIW, what you are proposing is exactly how some SHA variants work. For example SHA-224 is often implemented by generating SHA-256 and chopping off 4 bytes. So you can simply claim you're using SHA-248
            – slebetman
            Nov 26 at 9:57






          • 8




            @slebetman, IIRC, SHA-224 uses different initialization constants, too, so it doesn't give the same results as SHA-256 chopped off to length. And no, you can't say "you're using SHA-248", since no such thing exists in any standards. You could say you're using SHA-256 truncated to 248 bits, since, well, that's what you are doing.
            – ilkkachu
            Nov 26 at 11:19














          • 1




            Thanks very much! Quick question, why did you subtract 256 by 32 when calculating the generic birthday attack time? I'm decreasing the output size by 1 byte (8 bits) not 32 bits.
            – Aakil Fernandes
            Nov 25 at 22:04






          • 2




            it should be 8, my mistake. let me correct.
            – kelalaka
            Nov 25 at 22:05








          • 3




            @kelalaka - FWIW, what you are proposing is exactly how some SHA variants work. For example SHA-224 is often implemented by generating SHA-256 and chopping off 4 bytes. So you can simply claim you're using SHA-248
            – slebetman
            Nov 26 at 9:57






          • 8




            @slebetman, IIRC, SHA-224 uses different initialization constants, too, so it doesn't give the same results as SHA-256 chopped off to length. And no, you can't say "you're using SHA-248", since no such thing exists in any standards. You could say you're using SHA-256 truncated to 248 bits, since, well, that's what you are doing.
            – ilkkachu
            Nov 26 at 11:19








          1




          1




          Thanks very much! Quick question, why did you subtract 256 by 32 when calculating the generic birthday attack time? I'm decreasing the output size by 1 byte (8 bits) not 32 bits.
          – Aakil Fernandes
          Nov 25 at 22:04




          Thanks very much! Quick question, why did you subtract 256 by 32 when calculating the generic birthday attack time? I'm decreasing the output size by 1 byte (8 bits) not 32 bits.
          – Aakil Fernandes
          Nov 25 at 22:04




          2




          2




          it should be 8, my mistake. let me correct.
          – kelalaka
          Nov 25 at 22:05






          it should be 8, my mistake. let me correct.
          – kelalaka
          Nov 25 at 22:05






          3




          3




          @kelalaka - FWIW, what you are proposing is exactly how some SHA variants work. For example SHA-224 is often implemented by generating SHA-256 and chopping off 4 bytes. So you can simply claim you're using SHA-248
          – slebetman
          Nov 26 at 9:57




          @kelalaka - FWIW, what you are proposing is exactly how some SHA variants work. For example SHA-224 is often implemented by generating SHA-256 and chopping off 4 bytes. So you can simply claim you're using SHA-248
          – slebetman
          Nov 26 at 9:57




          8




          8




          @slebetman, IIRC, SHA-224 uses different initialization constants, too, so it doesn't give the same results as SHA-256 chopped off to length. And no, you can't say "you're using SHA-248", since no such thing exists in any standards. You could say you're using SHA-256 truncated to 248 bits, since, well, that's what you are doing.
          – ilkkachu
          Nov 26 at 11:19




          @slebetman, IIRC, SHA-224 uses different initialization constants, too, so it doesn't give the same results as SHA-256 chopped off to length. And no, you can't say "you're using SHA-248", since no such thing exists in any standards. You could say you're using SHA-256 truncated to 248 bits, since, well, that's what you are doing.
          – ilkkachu
          Nov 26 at 11:19


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Cryptography Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcrypto.stackexchange.com%2fquestions%2f64314%2fwhat-are-the-consequences-of-removing-a-single-byte-from-a-sha256-hash%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          How to change which sound is reproduced for terminal bell?

          Title Spacing in Bjornstrup Chapter, Removing Chapter Number From Contents

          Can I use Tabulator js library in my java Spring + Thymeleaf project?