How does the MQTT server output a single raw file from multiple client publications?
I'm trying my first IoT project whereby I want to:
- have an electret microphone capture audio
- have an ESP8266 NodeMCU 12-E board submit captured audio to a remote server
- have a remote server receive the audio data using an MQTT server
- publish the audio data as a .WAV file on the server
I saw someone online demonstrate something similar in this YouTube video
https://www.youtube.com/watch?v=rU_Pw9Jb_PM
The author shared the project on github here
https://github.com/hjltu/esp8266-wifi-microphone
When I study the code, I think what I see is the author taking the value of analogRead(A) and appending it to some kind of string as a payload, which is then published to an MQTT server.
I see that the author expects the MQTT server or some other software to process the ESP8266 microphone audio data and output it as a .RAW file. This RAW file is eventually converted to a .WAV file with the help of ffmpeg.
My question is this: What command allows MQTT server to generate the .RAW file? Or is this done by an entirely different software? And it appears to me that for a single recording/audio file, the my_record() of esp8266-wifi-mic.ino file will send multiple payloads to the MQTT server. So how does the MQTT server know which published transmission belongs to which RAW file?
mqtt esp8266
asked Jan 26 at 23:26
JohnJohn
1134
add a comment |
I'm trying my first IoT project whereby I want to:
- have an electret microphone capture audio
- have an ESP8266 NodeMCU 12-E board submit captured audio to a remote server
- have a remote server receive the audio data using an MQTT server
- publish the audio data as a .WAV file on the server
I saw someone online demonstrate something similar in this YouTube video
https://www.youtube.com/watch?v=rU_Pw9Jb_PM
The author shared the project on github here
https://github.com/hjltu/esp8266-wifi-microphone
When I study the code, I think what I see is the author taking the value of analogRead(A) and appending it to some kind of string as a payload, which is then published to an MQTT server.
I see that the author expects the MQTT server or some other software to process the ESP8266 microphone audio data and output it as a .RAW file. This RAW file is eventually converted to a .WAV file with the help of ffmpeg.
My question is this: What command allows MQTT server to generate the .RAW file? Or is this done by an entirely different software? And it appears to me that for a single recording/audio file, the my_record() of esp8266-wifi-mic.ino file will send multiple payloads to the MQTT server. So how does the MQTT server know which published transmission belongs to which RAW file?
mqtt esp8266
asked Jan 26 at 23:26
JohnJohn
1134
There no such thing as a MQTT server, it's a MQTT broker. Brokers don't do any processing of data, messages are received by another MQTT client that subscribed to the topic that messages are published on
– hardillb
Jan 27 at 7:36
add a comment |
I'm trying my first IoT project whereby I want to:
- have an electret microphone capture audio
- have an ESP8266 NodeMCU 12-E board submit captured audio to a remote server
- have a remote server receive the audio data using an MQTT server
- publish the audio data as a .WAV file on the server
I saw someone online demonstrate something similar in this YouTube video
https://www.youtube.com/watch?v=rU_Pw9Jb_PM
The author shared the project on github here
https://github.com/hjltu/esp8266-wifi-microphone
When I study the code, I think what I see is the author taking the value of analogRead(A) and appending it to some kind of string as a payload, which is then published to an MQTT server.
I see that the author expects the MQTT server or some other software to process the ESP8266 microphone audio data and output it as a .RAW file. This RAW file is eventually converted to a .WAV file with the help of ffmpeg.
My question is this: What command allows MQTT server to generate the .RAW file? Or is this done by an entirely different software? And it appears to me that for a single recording/audio file, the my_record() of esp8266-wifi-mic.ino file will send multiple payloads to the MQTT server. So how does the MQTT server know which published transmission belongs to which RAW file?
mqtt esp8266
asked Jan 26 at 23:26
JohnJohn
1134
I'm trying my first IoT project whereby I want to:
- have an electret microphone capture audio
- have an ESP8266 NodeMCU 12-E board submit captured audio to a remote server
- have a remote server receive the audio data using an MQTT server
- publish the audio data as a .WAV file on the server
I saw someone online demonstrate something similar in this YouTube video
https://www.youtube.com/watch?v=rU_Pw9Jb_PM
The author shared the project on github here
https://github.com/hjltu/esp8266-wifi-microphone
When I study the code, I think what I see is the author taking the value of analogRead(A) and appending it to some kind of string as a payload, which is then published to an MQTT server.
I see that the author expects the MQTT server or some other software to process the ESP8266 microphone audio data and output it as a .RAW file. This RAW file is eventually converted to a .WAV file with the help of ffmpeg.
My question is this: What command allows MQTT server to generate the .RAW file? Or is this done by an entirely different software? And it appears to me that for a single recording/audio file, the my_record() of esp8266-wifi-mic.ino file will send multiple payloads to the MQTT server. So how does the MQTT server know which published transmission belongs to which RAW file?
mqtt esp8266
mqtt esp8266
asked Jan 26 at 23:26
JohnJohn
1134
asked Jan 26 at 23:26
JohnJohn
1134
asked Jan 26 at 23:26
JohnJohn
1134
asked Jan 26 at 23:26
JohnJohn
1134
asked Jan 26 at 23:26
JohnJohn
1134
1134
There no such thing as a MQTT server, it's a MQTT broker. Brokers don't do any processing of data, messages are received by another MQTT client that subscribed to the topic that messages are published on
– hardillb
Jan 27 at 7:36
add a comment |
There no such thing as a MQTT server, it's a MQTT broker. Brokers don't do any processing of data, messages are received by another MQTT client that subscribed to the topic that messages are published on
– hardillb
Jan 27 at 7:36
There no such thing as a MQTT server, it's a MQTT broker. Brokers don't do any processing of data, messages are received by another MQTT client that subscribed to the topic that messages are published on
– hardillb
Jan 27 at 7:36
There no such thing as a MQTT server, it's a MQTT broker. Brokers don't do any processing of data, messages are received by another MQTT client that subscribed to the topic that messages are published on
– hardillb
Jan 27 at 7:36
add a comment |
1 Answer
1
active
oldest
votes
You are correct about the microphone input, void my_record() samples the microphone output level 1000 times, appending each reading to a string variable and publishes the resulting string to an MQTT broker.
This process repeats 11 times every time my_record() is called.
note: You are sort of misunderstanding about the .RAW file. It is a raw file, meaning that it is unprocessed and unformatted .... just a stream of bytes. Using the term .RAW implies a file name extension.
What command allows MQTT server to generate the .RAW file?
The MQTT broker (server) does not generate the raw file, the file is published to the MQTT broker by an outside source, the ESP8266 in this instance.
So how does the MQTT server know which published transmission belongs to which RAW file?
It does not know. All it does is to relay messages. It is up to the publisher to send to the correct topic and it is up to the subscriber to watch the data at the correct topic.
The messages could arrive to the subscriber out of sequence, so a sequence number needs to be included with the message if a correct data sequence is desired.
Have a look at these for a visual demo of MQTT messages.
https://shiftr.io/shiftr-io/demo ... you can only subscribe to this one
https://shiftr.io/try or https://shiftr.io/shiftr-io/try ... you can publish to this one (and subscribe)
You can get your own account and watch your own messages being sent and received without the clutter of other messages.
answered Jan 26 at 23:54
jsotolajsotola
1566
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
Jan 26 at 23:58
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
Jan 27 at 0:04
data can arrive out of sequence though ..... updated answer
– jsotola
Jan 27 at 0:09
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "666"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fiot.stackexchange.com%2fquestions%2f3838%2fhow-does-the-mqtt-server-output-a-single-raw-file-from-multiple-client-publicati%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You are correct about the microphone input, void my_record() samples the microphone output level 1000 times, appending each reading to a string variable and publishes the resulting string to an MQTT broker.
This process repeats 11 times every time my_record() is called.
note: You are sort of misunderstanding about the .RAW file. It is a raw file, meaning that it is unprocessed and unformatted .... just a stream of bytes. Using the term .RAW implies a file name extension.
What command allows MQTT server to generate the .RAW file?
The MQTT broker (server) does not generate the raw file, the file is published to the MQTT broker by an outside source, the ESP8266 in this instance.
So how does the MQTT server know which published transmission belongs to which RAW file?
It does not know. All it does is to relay messages. It is up to the publisher to send to the correct topic and it is up to the subscriber to watch the data at the correct topic.
The messages could arrive to the subscriber out of sequence, so a sequence number needs to be included with the message if a correct data sequence is desired.
Have a look at these for a visual demo of MQTT messages.
https://shiftr.io/shiftr-io/demo ... you can only subscribe to this one
https://shiftr.io/try or https://shiftr.io/shiftr-io/try ... you can publish to this one (and subscribe)
You can get your own account and watch your own messages being sent and received without the clutter of other messages.
answered Jan 26 at 23:54
jsotolajsotola
1566
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
Jan 26 at 23:58
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
Jan 27 at 0:04
data can arrive out of sequence though ..... updated answer
– jsotola
Jan 27 at 0:09
add a comment |
You are correct about the microphone input, void my_record() samples the microphone output level 1000 times, appending each reading to a string variable and publishes the resulting string to an MQTT broker.
This process repeats 11 times every time my_record() is called.
note: You are sort of misunderstanding about the .RAW file. It is a raw file, meaning that it is unprocessed and unformatted .... just a stream of bytes. Using the term .RAW implies a file name extension.
What command allows MQTT server to generate the .RAW file?
The MQTT broker (server) does not generate the raw file, the file is published to the MQTT broker by an outside source, the ESP8266 in this instance.
So how does the MQTT server know which published transmission belongs to which RAW file?
It does not know. All it does is to relay messages. It is up to the publisher to send to the correct topic and it is up to the subscriber to watch the data at the correct topic.
The messages could arrive to the subscriber out of sequence, so a sequence number needs to be included with the message if a correct data sequence is desired.
Have a look at these for a visual demo of MQTT messages.
https://shiftr.io/shiftr-io/demo ... you can only subscribe to this one
https://shiftr.io/try or https://shiftr.io/shiftr-io/try ... you can publish to this one (and subscribe)
You can get your own account and watch your own messages being sent and received without the clutter of other messages.
answered Jan 26 at 23:54
jsotolajsotola
1566
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
Jan 26 at 23:58
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
Jan 27 at 0:04
data can arrive out of sequence though ..... updated answer
– jsotola
Jan 27 at 0:09
add a comment |
You are correct about the microphone input, void my_record() samples the microphone output level 1000 times, appending each reading to a string variable and publishes the resulting string to an MQTT broker.
This process repeats 11 times every time my_record() is called.
note: You are sort of misunderstanding about the .RAW file. It is a raw file, meaning that it is unprocessed and unformatted .... just a stream of bytes. Using the term .RAW implies a file name extension.
What command allows MQTT server to generate the .RAW file?
The MQTT broker (server) does not generate the raw file, the file is published to the MQTT broker by an outside source, the ESP8266 in this instance.
So how does the MQTT server know which published transmission belongs to which RAW file?
It does not know. All it does is to relay messages. It is up to the publisher to send to the correct topic and it is up to the subscriber to watch the data at the correct topic.
The messages could arrive to the subscriber out of sequence, so a sequence number needs to be included with the message if a correct data sequence is desired.
Have a look at these for a visual demo of MQTT messages.
https://shiftr.io/shiftr-io/demo ... you can only subscribe to this one
https://shiftr.io/try or https://shiftr.io/shiftr-io/try ... you can publish to this one (and subscribe)
You can get your own account and watch your own messages being sent and received without the clutter of other messages.
answered Jan 26 at 23:54
jsotolajsotola
1566
You are correct about the microphone input, void my_record() samples the microphone output level 1000 times, appending each reading to a string variable and publishes the resulting string to an MQTT broker.
This process repeats 11 times every time my_record() is called.
note: You are sort of misunderstanding about the .RAW file. It is a raw file, meaning that it is unprocessed and unformatted .... just a stream of bytes. Using the term .RAW implies a file name extension.
What command allows MQTT server to generate the .RAW file?
The MQTT broker (server) does not generate the raw file, the file is published to the MQTT broker by an outside source, the ESP8266 in this instance.
So how does the MQTT server know which published transmission belongs to which RAW file?
It does not know. All it does is to relay messages. It is up to the publisher to send to the correct topic and it is up to the subscriber to watch the data at the correct topic.
The messages could arrive to the subscriber out of sequence, so a sequence number needs to be included with the message if a correct data sequence is desired.
Have a look at these for a visual demo of MQTT messages.
https://shiftr.io/shiftr-io/demo ... you can only subscribe to this one
https://shiftr.io/try or https://shiftr.io/shiftr-io/try ... you can publish to this one (and subscribe)
You can get your own account and watch your own messages being sent and received without the clutter of other messages.
answered Jan 26 at 23:54
jsotolajsotola
1566
edited Jan 27 at 0:07
answered Jan 26 at 23:54
jsotolajsotola
1566
answered Jan 26 at 23:54
jsotolajsotola
1566
answered Jan 26 at 23:54
jsotolajsotola
1566
1566
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
Jan 26 at 23:58
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
Jan 27 at 0:04
data can arrive out of sequence though ..... updated answer
– jsotola
Jan 27 at 0:09
add a comment |
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
Jan 26 at 23:58
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
Jan 27 at 0:04
data can arrive out of sequence though ..... updated answer
– jsotola
Jan 27 at 0:09
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
Jan 26 at 23:58
Oh....I see. So that means I need to create another client application to read subscription data from MQTT server, and save content to a .raw file?
– John
Jan 26 at 23:58
1
1
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
Jan 27 at 0:04
exactly correct ..... the MQTT server is just a long pipe for your data
– jsotola
Jan 27 at 0:04
data can arrive out of sequence though ..... updated answer
– jsotola
Jan 27 at 0:09
data can arrive out of sequence though ..... updated answer
– jsotola
Jan 27 at 0:09
add a comment |
Thanks for contributing an answer to Internet of Things Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fiot.stackexchange.com%2fquestions%2f3838%2fhow-does-the-mqtt-server-output-a-single-raw-file-from-multiple-client-publicati%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
There no such thing as a MQTT server, it's a MQTT broker. Brokers don't do any processing of data, messages are received by another MQTT client that subscribed to the topic that messages are published on
– hardillb
Jan 27 at 7:36