How do I round datetime column to nearest quarter hour
I have loaded a data file into a Python pandas dataframe. I has a datetime column of the format 2015-07-18 13:53:33.280
.
What I need to do is create a new column that rounds this out to its nearest quarter hour. So, the date above will be rounded to 2015-07-18 13:45:00.000
.
How do I do this in pandas? I tried using the solution from here, but get an 'Series' object has no attribute 'year'
error.
python datetime pandas python-datetime
add a comment |
I have loaded a data file into a Python pandas dataframe. I has a datetime column of the format 2015-07-18 13:53:33.280
.
What I need to do is create a new column that rounds this out to its nearest quarter hour. So, the date above will be rounded to 2015-07-18 13:45:00.000
.
How do I do this in pandas? I tried using the solution from here, but get an 'Series' object has no attribute 'year'
error.
python datetime pandas python-datetime
add a comment |
I have loaded a data file into a Python pandas dataframe. I has a datetime column of the format 2015-07-18 13:53:33.280
.
What I need to do is create a new column that rounds this out to its nearest quarter hour. So, the date above will be rounded to 2015-07-18 13:45:00.000
.
How do I do this in pandas? I tried using the solution from here, but get an 'Series' object has no attribute 'year'
error.
python datetime pandas python-datetime
I have loaded a data file into a Python pandas dataframe. I has a datetime column of the format 2015-07-18 13:53:33.280
.
What I need to do is create a new column that rounds this out to its nearest quarter hour. So, the date above will be rounded to 2015-07-18 13:45:00.000
.
How do I do this in pandas? I tried using the solution from here, but get an 'Series' object has no attribute 'year'
error.
python datetime pandas python-datetime
python datetime pandas python-datetime
edited May 23 '17 at 10:30
Community♦
11
11
asked Sep 2 '15 at 4:13
sfactor
4,1812072133
4,1812072133
add a comment |
add a comment |
4 Answers
4
active
oldest
votes
Assuming that your series is made up of datetime
objects, You need to use Series.apply
. Example -
import datetime
df['<column>'] = df['<column>'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,15*(dt.minute // 15)))
The above example to always round to the previous quarter hour (behavior similar to floor function).
EDIT
To round to the correct quarter hour (as in , if its 7 mins 30 seconds past previous quarter, to show the next quarter) . We can use the below example -
import datetime
df['<column>'] = df['<column>'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,15*round((float(dt.minute) + float(dt.second)/60) / 15)))
The above would only take the latest seconds into consideration , if you want the millisecond/microsecond into consideration , you can add that to the above equation as - (float(dt.minute) + float(dt.second)/60 + float(dt.microsecond)/60000000)
I believe this answer is actually incorrect - since it will always round down to the quarter hour before it, instead of to the nearest quarter hour.
– Thomas Mo
Aug 2 '17 at 17:38
@ThomasMo updated the answer for that as well. Previous behavior would have been similar tofloor
behavior
– Anand S Kumar
Aug 4 '17 at 5:41
add a comment |
You can use round(freq)
. There is also a shortcut column.dt
for datetime functions access (as @laurens-koppenol suggests).
Here's one-liner:
df['old column'].dt.round('15min')
String aliases for valid frequencies can be found here. Full working example:
In [1]: import pandas as pd
In [2]: df = pd.DataFrame([pd.Timestamp('2015-07-18 13:53:33.280'),
pd.Timestamp('2015-07-18 13:33:33.330')],
columns=['old column'])
In [3]: df['new column']=df['old column'].dt.round('15min')
In [4]: df
Out[4]:
old column new column
0 2015-07-18 13:53:33.280 2015-07-18 14:00:00
1 2015-07-18 13:33:33.330 2015-07-18 13:30:00
4
If one want to round to the closest time below or above it is possible to usefloor
andceil
, respectively. See code
– Dror
Aug 3 '17 at 11:55
add a comment |
This looks a little nicer
column.dt.
allows the datetime functions for datetime columns, like column.str.
does for string-like columns
datetime-like properties API reference
import pandas as pd
# test df
df = pd.DataFrame([{'old_column':pd.Timestamp('2015-07-18 13:53:33.280')}])
df['new_column'] = df['old_column'].dt.round('15min')
df
add a comment |
Anand S Kumar's answer doesn't round to the nearest quarter hour, it cuts off the minutes to the nearest 15 minutes below it.
Actually, in your example 2015-07-18 13:53:33.280
should round to 2015-07-18 14:00:00.000
since 53:33.280
is closer to 60 minutes than 45 minutes.
I found an more robust answer for rounding in this post.
For your situation this should work:
import datetime
def round_time(time, round_to):
"""roundTo is the number of minutes to round to"""
rounded = time + datetime.timedelta(minutes=round_to/2.)
rounded -= datetime.timedelta(minutes=rounded.minute % round_to,
seconds=rounded.second,
microseconds=rounded.microsecond)
return rounded
dt['dtcolumn'] = df['dtcolumn'].apply(lambda x: round_time(x))
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f32344533%2fhow-do-i-round-datetime-column-to-nearest-quarter-hour%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
Assuming that your series is made up of datetime
objects, You need to use Series.apply
. Example -
import datetime
df['<column>'] = df['<column>'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,15*(dt.minute // 15)))
The above example to always round to the previous quarter hour (behavior similar to floor function).
EDIT
To round to the correct quarter hour (as in , if its 7 mins 30 seconds past previous quarter, to show the next quarter) . We can use the below example -
import datetime
df['<column>'] = df['<column>'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,15*round((float(dt.minute) + float(dt.second)/60) / 15)))
The above would only take the latest seconds into consideration , if you want the millisecond/microsecond into consideration , you can add that to the above equation as - (float(dt.minute) + float(dt.second)/60 + float(dt.microsecond)/60000000)
I believe this answer is actually incorrect - since it will always round down to the quarter hour before it, instead of to the nearest quarter hour.
– Thomas Mo
Aug 2 '17 at 17:38
@ThomasMo updated the answer for that as well. Previous behavior would have been similar tofloor
behavior
– Anand S Kumar
Aug 4 '17 at 5:41
add a comment |
Assuming that your series is made up of datetime
objects, You need to use Series.apply
. Example -
import datetime
df['<column>'] = df['<column>'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,15*(dt.minute // 15)))
The above example to always round to the previous quarter hour (behavior similar to floor function).
EDIT
To round to the correct quarter hour (as in , if its 7 mins 30 seconds past previous quarter, to show the next quarter) . We can use the below example -
import datetime
df['<column>'] = df['<column>'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,15*round((float(dt.minute) + float(dt.second)/60) / 15)))
The above would only take the latest seconds into consideration , if you want the millisecond/microsecond into consideration , you can add that to the above equation as - (float(dt.minute) + float(dt.second)/60 + float(dt.microsecond)/60000000)
I believe this answer is actually incorrect - since it will always round down to the quarter hour before it, instead of to the nearest quarter hour.
– Thomas Mo
Aug 2 '17 at 17:38
@ThomasMo updated the answer for that as well. Previous behavior would have been similar tofloor
behavior
– Anand S Kumar
Aug 4 '17 at 5:41
add a comment |
Assuming that your series is made up of datetime
objects, You need to use Series.apply
. Example -
import datetime
df['<column>'] = df['<column>'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,15*(dt.minute // 15)))
The above example to always round to the previous quarter hour (behavior similar to floor function).
EDIT
To round to the correct quarter hour (as in , if its 7 mins 30 seconds past previous quarter, to show the next quarter) . We can use the below example -
import datetime
df['<column>'] = df['<column>'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,15*round((float(dt.minute) + float(dt.second)/60) / 15)))
The above would only take the latest seconds into consideration , if you want the millisecond/microsecond into consideration , you can add that to the above equation as - (float(dt.minute) + float(dt.second)/60 + float(dt.microsecond)/60000000)
Assuming that your series is made up of datetime
objects, You need to use Series.apply
. Example -
import datetime
df['<column>'] = df['<column>'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,15*(dt.minute // 15)))
The above example to always round to the previous quarter hour (behavior similar to floor function).
EDIT
To round to the correct quarter hour (as in , if its 7 mins 30 seconds past previous quarter, to show the next quarter) . We can use the below example -
import datetime
df['<column>'] = df['<column>'].apply(lambda dt: datetime.datetime(dt.year, dt.month, dt.day, dt.hour,15*round((float(dt.minute) + float(dt.second)/60) / 15)))
The above would only take the latest seconds into consideration , if you want the millisecond/microsecond into consideration , you can add that to the above equation as - (float(dt.minute) + float(dt.second)/60 + float(dt.microsecond)/60000000)
edited Aug 4 '17 at 5:38
answered Sep 2 '15 at 4:25
Anand S Kumar
58.1k1292112
58.1k1292112
I believe this answer is actually incorrect - since it will always round down to the quarter hour before it, instead of to the nearest quarter hour.
– Thomas Mo
Aug 2 '17 at 17:38
@ThomasMo updated the answer for that as well. Previous behavior would have been similar tofloor
behavior
– Anand S Kumar
Aug 4 '17 at 5:41
add a comment |
I believe this answer is actually incorrect - since it will always round down to the quarter hour before it, instead of to the nearest quarter hour.
– Thomas Mo
Aug 2 '17 at 17:38
@ThomasMo updated the answer for that as well. Previous behavior would have been similar tofloor
behavior
– Anand S Kumar
Aug 4 '17 at 5:41
I believe this answer is actually incorrect - since it will always round down to the quarter hour before it, instead of to the nearest quarter hour.
– Thomas Mo
Aug 2 '17 at 17:38
I believe this answer is actually incorrect - since it will always round down to the quarter hour before it, instead of to the nearest quarter hour.
– Thomas Mo
Aug 2 '17 at 17:38
@ThomasMo updated the answer for that as well. Previous behavior would have been similar to
floor
behavior– Anand S Kumar
Aug 4 '17 at 5:41
@ThomasMo updated the answer for that as well. Previous behavior would have been similar to
floor
behavior– Anand S Kumar
Aug 4 '17 at 5:41
add a comment |
You can use round(freq)
. There is also a shortcut column.dt
for datetime functions access (as @laurens-koppenol suggests).
Here's one-liner:
df['old column'].dt.round('15min')
String aliases for valid frequencies can be found here. Full working example:
In [1]: import pandas as pd
In [2]: df = pd.DataFrame([pd.Timestamp('2015-07-18 13:53:33.280'),
pd.Timestamp('2015-07-18 13:33:33.330')],
columns=['old column'])
In [3]: df['new column']=df['old column'].dt.round('15min')
In [4]: df
Out[4]:
old column new column
0 2015-07-18 13:53:33.280 2015-07-18 14:00:00
1 2015-07-18 13:33:33.330 2015-07-18 13:30:00
4
If one want to round to the closest time below or above it is possible to usefloor
andceil
, respectively. See code
– Dror
Aug 3 '17 at 11:55
add a comment |
You can use round(freq)
. There is also a shortcut column.dt
for datetime functions access (as @laurens-koppenol suggests).
Here's one-liner:
df['old column'].dt.round('15min')
String aliases for valid frequencies can be found here. Full working example:
In [1]: import pandas as pd
In [2]: df = pd.DataFrame([pd.Timestamp('2015-07-18 13:53:33.280'),
pd.Timestamp('2015-07-18 13:33:33.330')],
columns=['old column'])
In [3]: df['new column']=df['old column'].dt.round('15min')
In [4]: df
Out[4]:
old column new column
0 2015-07-18 13:53:33.280 2015-07-18 14:00:00
1 2015-07-18 13:33:33.330 2015-07-18 13:30:00
4
If one want to round to the closest time below or above it is possible to usefloor
andceil
, respectively. See code
– Dror
Aug 3 '17 at 11:55
add a comment |
You can use round(freq)
. There is also a shortcut column.dt
for datetime functions access (as @laurens-koppenol suggests).
Here's one-liner:
df['old column'].dt.round('15min')
String aliases for valid frequencies can be found here. Full working example:
In [1]: import pandas as pd
In [2]: df = pd.DataFrame([pd.Timestamp('2015-07-18 13:53:33.280'),
pd.Timestamp('2015-07-18 13:33:33.330')],
columns=['old column'])
In [3]: df['new column']=df['old column'].dt.round('15min')
In [4]: df
Out[4]:
old column new column
0 2015-07-18 13:53:33.280 2015-07-18 14:00:00
1 2015-07-18 13:33:33.330 2015-07-18 13:30:00
You can use round(freq)
. There is also a shortcut column.dt
for datetime functions access (as @laurens-koppenol suggests).
Here's one-liner:
df['old column'].dt.round('15min')
String aliases for valid frequencies can be found here. Full working example:
In [1]: import pandas as pd
In [2]: df = pd.DataFrame([pd.Timestamp('2015-07-18 13:53:33.280'),
pd.Timestamp('2015-07-18 13:33:33.330')],
columns=['old column'])
In [3]: df['new column']=df['old column'].dt.round('15min')
In [4]: df
Out[4]:
old column new column
0 2015-07-18 13:53:33.280 2015-07-18 14:00:00
1 2015-07-18 13:33:33.330 2015-07-18 13:30:00
edited Jan 31 '18 at 22:48
answered Oct 31 '16 at 12:24
tworec
1,81111323
1,81111323
4
If one want to round to the closest time below or above it is possible to usefloor
andceil
, respectively. See code
– Dror
Aug 3 '17 at 11:55
add a comment |
4
If one want to round to the closest time below or above it is possible to usefloor
andceil
, respectively. See code
– Dror
Aug 3 '17 at 11:55
4
4
If one want to round to the closest time below or above it is possible to use
floor
and ceil
, respectively. See code– Dror
Aug 3 '17 at 11:55
If one want to round to the closest time below or above it is possible to use
floor
and ceil
, respectively. See code– Dror
Aug 3 '17 at 11:55
add a comment |
This looks a little nicer
column.dt.
allows the datetime functions for datetime columns, like column.str.
does for string-like columns
datetime-like properties API reference
import pandas as pd
# test df
df = pd.DataFrame([{'old_column':pd.Timestamp('2015-07-18 13:53:33.280')}])
df['new_column'] = df['old_column'].dt.round('15min')
df
add a comment |
This looks a little nicer
column.dt.
allows the datetime functions for datetime columns, like column.str.
does for string-like columns
datetime-like properties API reference
import pandas as pd
# test df
df = pd.DataFrame([{'old_column':pd.Timestamp('2015-07-18 13:53:33.280')}])
df['new_column'] = df['old_column'].dt.round('15min')
df
add a comment |
This looks a little nicer
column.dt.
allows the datetime functions for datetime columns, like column.str.
does for string-like columns
datetime-like properties API reference
import pandas as pd
# test df
df = pd.DataFrame([{'old_column':pd.Timestamp('2015-07-18 13:53:33.280')}])
df['new_column'] = df['old_column'].dt.round('15min')
df
This looks a little nicer
column.dt.
allows the datetime functions for datetime columns, like column.str.
does for string-like columns
datetime-like properties API reference
import pandas as pd
# test df
df = pd.DataFrame([{'old_column':pd.Timestamp('2015-07-18 13:53:33.280')}])
df['new_column'] = df['old_column'].dt.round('15min')
df
answered Feb 27 '17 at 13:56
Laurens Koppenol
1,097817
1,097817
add a comment |
add a comment |
Anand S Kumar's answer doesn't round to the nearest quarter hour, it cuts off the minutes to the nearest 15 minutes below it.
Actually, in your example 2015-07-18 13:53:33.280
should round to 2015-07-18 14:00:00.000
since 53:33.280
is closer to 60 minutes than 45 minutes.
I found an more robust answer for rounding in this post.
For your situation this should work:
import datetime
def round_time(time, round_to):
"""roundTo is the number of minutes to round to"""
rounded = time + datetime.timedelta(minutes=round_to/2.)
rounded -= datetime.timedelta(minutes=rounded.minute % round_to,
seconds=rounded.second,
microseconds=rounded.microsecond)
return rounded
dt['dtcolumn'] = df['dtcolumn'].apply(lambda x: round_time(x))
add a comment |
Anand S Kumar's answer doesn't round to the nearest quarter hour, it cuts off the minutes to the nearest 15 minutes below it.
Actually, in your example 2015-07-18 13:53:33.280
should round to 2015-07-18 14:00:00.000
since 53:33.280
is closer to 60 minutes than 45 minutes.
I found an more robust answer for rounding in this post.
For your situation this should work:
import datetime
def round_time(time, round_to):
"""roundTo is the number of minutes to round to"""
rounded = time + datetime.timedelta(minutes=round_to/2.)
rounded -= datetime.timedelta(minutes=rounded.minute % round_to,
seconds=rounded.second,
microseconds=rounded.microsecond)
return rounded
dt['dtcolumn'] = df['dtcolumn'].apply(lambda x: round_time(x))
add a comment |
Anand S Kumar's answer doesn't round to the nearest quarter hour, it cuts off the minutes to the nearest 15 minutes below it.
Actually, in your example 2015-07-18 13:53:33.280
should round to 2015-07-18 14:00:00.000
since 53:33.280
is closer to 60 minutes than 45 minutes.
I found an more robust answer for rounding in this post.
For your situation this should work:
import datetime
def round_time(time, round_to):
"""roundTo is the number of minutes to round to"""
rounded = time + datetime.timedelta(minutes=round_to/2.)
rounded -= datetime.timedelta(minutes=rounded.minute % round_to,
seconds=rounded.second,
microseconds=rounded.microsecond)
return rounded
dt['dtcolumn'] = df['dtcolumn'].apply(lambda x: round_time(x))
Anand S Kumar's answer doesn't round to the nearest quarter hour, it cuts off the minutes to the nearest 15 minutes below it.
Actually, in your example 2015-07-18 13:53:33.280
should round to 2015-07-18 14:00:00.000
since 53:33.280
is closer to 60 minutes than 45 minutes.
I found an more robust answer for rounding in this post.
For your situation this should work:
import datetime
def round_time(time, round_to):
"""roundTo is the number of minutes to round to"""
rounded = time + datetime.timedelta(minutes=round_to/2.)
rounded -= datetime.timedelta(minutes=rounded.minute % round_to,
seconds=rounded.second,
microseconds=rounded.microsecond)
return rounded
dt['dtcolumn'] = df['dtcolumn'].apply(lambda x: round_time(x))
edited May 23 '17 at 12:18
Community♦
11
11
answered Sep 1 '16 at 22:16
Eric Blum
311412
311412
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f32344533%2fhow-do-i-round-datetime-column-to-nearest-quarter-hour%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown