P-values for Spark Logistic Regression
up vote
0
down vote
favorite
I am trying to find the p-values for coefficients for Spark 2.3 Logistic Regression. Specifically, if I do:
lr = LogisticRegression(labelCol="clicks", featuresCol="features", maxIter=10,elasticNetParam=1)
# Train model with Training Data
lrModel = lr.fit(trainingData)
print(lrModel.coefficients) #get coefficients
will return a list of the coefficient values, but without a p-value or confidence interval. I could not find in the documentation (https://spark.apache.org/docs/latest/api/python/pyspark.ml.html) where p-values for coefficients could be found.
Similir Questions that do not answer what I am looking for:
How to calculate p-values in Spark's Logistic Regression?
(This is an older version of Spark, and I could not confirm if it actually returns a pvalue; also note that that function does not take in pipelines (e.g. one hot encoding pipelines etc.))
apache-spark databricks
asked Nov 13 at 2:53
Jonathan
498
add a comment |
up vote
0
down vote
favorite
I am trying to find the p-values for coefficients for Spark 2.3 Logistic Regression. Specifically, if I do:
lr = LogisticRegression(labelCol="clicks", featuresCol="features", maxIter=10,elasticNetParam=1)
# Train model with Training Data
lrModel = lr.fit(trainingData)
print(lrModel.coefficients) #get coefficients
will return a list of the coefficient values, but without a p-value or confidence interval. I could not find in the documentation (https://spark.apache.org/docs/latest/api/python/pyspark.ml.html) where p-values for coefficients could be found.
Similir Questions that do not answer what I am looking for:
How to calculate p-values in Spark's Logistic Regression?
(This is an older version of Spark, and I could not confirm if it actually returns a pvalue; also note that that function does not take in pipelines (e.g. one hot encoding pipelines etc.))
apache-spark databricks
asked Nov 13 at 2:53
Jonathan
498
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am trying to find the p-values for coefficients for Spark 2.3 Logistic Regression. Specifically, if I do:
lr = LogisticRegression(labelCol="clicks", featuresCol="features", maxIter=10,elasticNetParam=1)
# Train model with Training Data
lrModel = lr.fit(trainingData)
print(lrModel.coefficients) #get coefficients
will return a list of the coefficient values, but without a p-value or confidence interval. I could not find in the documentation (https://spark.apache.org/docs/latest/api/python/pyspark.ml.html) where p-values for coefficients could be found.
Similir Questions that do not answer what I am looking for:
How to calculate p-values in Spark's Logistic Regression?
(This is an older version of Spark, and I could not confirm if it actually returns a pvalue; also note that that function does not take in pipelines (e.g. one hot encoding pipelines etc.))
apache-spark databricks
asked Nov 13 at 2:53
Jonathan
498
I am trying to find the p-values for coefficients for Spark 2.3 Logistic Regression. Specifically, if I do:
lr = LogisticRegression(labelCol="clicks", featuresCol="features", maxIter=10,elasticNetParam=1)
# Train model with Training Data
lrModel = lr.fit(trainingData)
print(lrModel.coefficients) #get coefficients
will return a list of the coefficient values, but without a p-value or confidence interval. I could not find in the documentation (https://spark.apache.org/docs/latest/api/python/pyspark.ml.html) where p-values for coefficients could be found.
Similir Questions that do not answer what I am looking for:
How to calculate p-values in Spark's Logistic Regression?
(This is an older version of Spark, and I could not confirm if it actually returns a pvalue; also note that that function does not take in pipelines (e.g. one hot encoding pipelines etc.))
apache-spark databricks
apache-spark databricks
asked Nov 13 at 2:53
Jonathan
498
asked Nov 13 at 2:53
Jonathan
498
edited Nov 13 at 3:05
asked Nov 13 at 2:53
Jonathan
498
asked Nov 13 at 2:53
Jonathan
498
asked Nov 13 at 2:53
Jonathan
498
498
add a comment |
add a comment |
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53273093%2fp-values-for-spark-logistic-regression%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
