Couchbase Disk Usage












1















I'm using Couchbase, with .net - Latest CouchbaseNetClient.



Couchbase Serer 4.6.2 Enterprise running.



I have an app where It performs 5k/ops.



The problem I am facing - is as soon as I start this app - then the CPU usage jumps to 100% and Disk Usage jumps to 100% also (as noted on the ctrl+alt+del ->Performance->Disk 0 (C:)



I have tested across a few PC.s - All are SSD (Samsung 850 PRO) and a dell M.2. Both have excellent r/w capabilities which I would have thought would swallow 5k ops.



The machine with the 850 PRO is a 6c/12t 980x CPU.
The machine with the m2.dell is a 4c/8t i76700hq.



I'm using a standard Upsert as per couchbase doco's. When I comment out this line, CPU goes right down to expected levels.



Is there some form of tweaking/performance/indexing mechanism I am missing entirely here??



Regards,



Chud



EDIT #1



So I uninstalled Couchbase on the Dell XPS and reinstalled it as memcached server only. Funny thing is, the CPU goes to 100% and now disk stays at 0% (expected of course). I have commented out the lines that actually do the Upsert, and CPU usage remains virtually unchanged to what it was before I started the app, even though the app itself is still doing a fair bit of crunching, it's just not entering the data into the (now) memcached database.



How can the upsert be so so so expensive? It's only a small Json file...



EDIT #2



So here is some samples that may get you going. I have changed the models around a some what to keep confidentiality in what I am doing, so it is "untested" so to speak. So if there are bugs in this I apologise, I literally just hacked it together.



Json Sample Below - I guess put this in a loop and let it rain, see if you can produce the same load.



{"Position":10,"Type":0,"Type2":5,"Value":0.0,"Type3":1,"Location":"Bla","Type":"Ex","Id":"11111"}

public class RootObject
{
public int Position { get; set; }
public MyFirstType Type { get; set; }
public MySecondType Type2 { get; set; }
public double Value { get; set; }
public MyThirdType Type3 { get; set; }
public string Location { get; set; }
public string Id { get; set; }
}

public enum MyFirstType
{
A,
B
}

public enum MySecondType
{
C,
D,
E,
F,
G,
H
}

public enum MyThirdType
{
I,
J
}
public class CouchBaseInstanceTest
{
private Cluster CouchClient;
private IBucket Bucket;

public CouchBaseInstanceTest()
{
CouchClient = new Cluster(new ClientConfiguration
{
Servers = new List<Uri> {new Uri("http://localhost:8091")}

});
// I think CB v 5.0 and over, or 5.5 needs auth against the bucket
//CouchClient.Authenticate("admin", "password");
Bucket = CouchClient.OpenBucket("default");
}
public bool Upsert(string key, dynamic value, double expiry)
{
var Document = new Document<dynamic>()
{
Id = key,
Content = value,
Expiry = TimeSpan.FromHours(expiry).ToTtl()
};
return Bucket.Upsert(Document).Success;
}
}


So as soon as that Bucket.Upsert is hit - the CPU/Disk maxes out.... or in just the memcached config....the CPU maxes out....



To be honest though, the code here is really irrelevant I think. When I replaced the document values with a blank string "" - The problem was still the same










share|improve this question

























  • Sounds like you should ask this on github or where ever this is located.

    – TheGeneral
    Nov 19 '18 at 6:28











  • Could you post the minimum code to recreate this? It's possible that you're creating too many connections or something like that.

    – Matthew Groves
    Nov 20 '18 at 15:01











  • Hi Matthew, I actually use a DI Container to establish the connection. This is only done once on program startup, and then the only thing called is an Upsert Function. I have even tried to use the Upsert(Dictionary<string,dynamic> "bulk" type function, but it makes no difference.

    – The_Chud
    Nov 20 '18 at 22:31
















1















I'm using Couchbase, with .net - Latest CouchbaseNetClient.



Couchbase Serer 4.6.2 Enterprise running.



I have an app where It performs 5k/ops.



The problem I am facing - is as soon as I start this app - then the CPU usage jumps to 100% and Disk Usage jumps to 100% also (as noted on the ctrl+alt+del ->Performance->Disk 0 (C:)



I have tested across a few PC.s - All are SSD (Samsung 850 PRO) and a dell M.2. Both have excellent r/w capabilities which I would have thought would swallow 5k ops.



The machine with the 850 PRO is a 6c/12t 980x CPU.
The machine with the m2.dell is a 4c/8t i76700hq.



I'm using a standard Upsert as per couchbase doco's. When I comment out this line, CPU goes right down to expected levels.



Is there some form of tweaking/performance/indexing mechanism I am missing entirely here??



Regards,



Chud



EDIT #1



So I uninstalled Couchbase on the Dell XPS and reinstalled it as memcached server only. Funny thing is, the CPU goes to 100% and now disk stays at 0% (expected of course). I have commented out the lines that actually do the Upsert, and CPU usage remains virtually unchanged to what it was before I started the app, even though the app itself is still doing a fair bit of crunching, it's just not entering the data into the (now) memcached database.



How can the upsert be so so so expensive? It's only a small Json file...



EDIT #2



So here is some samples that may get you going. I have changed the models around a some what to keep confidentiality in what I am doing, so it is "untested" so to speak. So if there are bugs in this I apologise, I literally just hacked it together.



Json Sample Below - I guess put this in a loop and let it rain, see if you can produce the same load.



{"Position":10,"Type":0,"Type2":5,"Value":0.0,"Type3":1,"Location":"Bla","Type":"Ex","Id":"11111"}

public class RootObject
{
public int Position { get; set; }
public MyFirstType Type { get; set; }
public MySecondType Type2 { get; set; }
public double Value { get; set; }
public MyThirdType Type3 { get; set; }
public string Location { get; set; }
public string Id { get; set; }
}

public enum MyFirstType
{
A,
B
}

public enum MySecondType
{
C,
D,
E,
F,
G,
H
}

public enum MyThirdType
{
I,
J
}
public class CouchBaseInstanceTest
{
private Cluster CouchClient;
private IBucket Bucket;

public CouchBaseInstanceTest()
{
CouchClient = new Cluster(new ClientConfiguration
{
Servers = new List<Uri> {new Uri("http://localhost:8091")}

});
// I think CB v 5.0 and over, or 5.5 needs auth against the bucket
//CouchClient.Authenticate("admin", "password");
Bucket = CouchClient.OpenBucket("default");
}
public bool Upsert(string key, dynamic value, double expiry)
{
var Document = new Document<dynamic>()
{
Id = key,
Content = value,
Expiry = TimeSpan.FromHours(expiry).ToTtl()
};
return Bucket.Upsert(Document).Success;
}
}


So as soon as that Bucket.Upsert is hit - the CPU/Disk maxes out.... or in just the memcached config....the CPU maxes out....



To be honest though, the code here is really irrelevant I think. When I replaced the document values with a blank string "" - The problem was still the same










share|improve this question

























  • Sounds like you should ask this on github or where ever this is located.

    – TheGeneral
    Nov 19 '18 at 6:28











  • Could you post the minimum code to recreate this? It's possible that you're creating too many connections or something like that.

    – Matthew Groves
    Nov 20 '18 at 15:01











  • Hi Matthew, I actually use a DI Container to establish the connection. This is only done once on program startup, and then the only thing called is an Upsert Function. I have even tried to use the Upsert(Dictionary<string,dynamic> "bulk" type function, but it makes no difference.

    – The_Chud
    Nov 20 '18 at 22:31














1












1








1








I'm using Couchbase, with .net - Latest CouchbaseNetClient.



Couchbase Serer 4.6.2 Enterprise running.



I have an app where It performs 5k/ops.



The problem I am facing - is as soon as I start this app - then the CPU usage jumps to 100% and Disk Usage jumps to 100% also (as noted on the ctrl+alt+del ->Performance->Disk 0 (C:)



I have tested across a few PC.s - All are SSD (Samsung 850 PRO) and a dell M.2. Both have excellent r/w capabilities which I would have thought would swallow 5k ops.



The machine with the 850 PRO is a 6c/12t 980x CPU.
The machine with the m2.dell is a 4c/8t i76700hq.



I'm using a standard Upsert as per couchbase doco's. When I comment out this line, CPU goes right down to expected levels.



Is there some form of tweaking/performance/indexing mechanism I am missing entirely here??



Regards,



Chud



EDIT #1



So I uninstalled Couchbase on the Dell XPS and reinstalled it as memcached server only. Funny thing is, the CPU goes to 100% and now disk stays at 0% (expected of course). I have commented out the lines that actually do the Upsert, and CPU usage remains virtually unchanged to what it was before I started the app, even though the app itself is still doing a fair bit of crunching, it's just not entering the data into the (now) memcached database.



How can the upsert be so so so expensive? It's only a small Json file...



EDIT #2



So here is some samples that may get you going. I have changed the models around a some what to keep confidentiality in what I am doing, so it is "untested" so to speak. So if there are bugs in this I apologise, I literally just hacked it together.



Json Sample Below - I guess put this in a loop and let it rain, see if you can produce the same load.



{"Position":10,"Type":0,"Type2":5,"Value":0.0,"Type3":1,"Location":"Bla","Type":"Ex","Id":"11111"}

public class RootObject
{
public int Position { get; set; }
public MyFirstType Type { get; set; }
public MySecondType Type2 { get; set; }
public double Value { get; set; }
public MyThirdType Type3 { get; set; }
public string Location { get; set; }
public string Id { get; set; }
}

public enum MyFirstType
{
A,
B
}

public enum MySecondType
{
C,
D,
E,
F,
G,
H
}

public enum MyThirdType
{
I,
J
}
public class CouchBaseInstanceTest
{
private Cluster CouchClient;
private IBucket Bucket;

public CouchBaseInstanceTest()
{
CouchClient = new Cluster(new ClientConfiguration
{
Servers = new List<Uri> {new Uri("http://localhost:8091")}

});
// I think CB v 5.0 and over, or 5.5 needs auth against the bucket
//CouchClient.Authenticate("admin", "password");
Bucket = CouchClient.OpenBucket("default");
}
public bool Upsert(string key, dynamic value, double expiry)
{
var Document = new Document<dynamic>()
{
Id = key,
Content = value,
Expiry = TimeSpan.FromHours(expiry).ToTtl()
};
return Bucket.Upsert(Document).Success;
}
}


So as soon as that Bucket.Upsert is hit - the CPU/Disk maxes out.... or in just the memcached config....the CPU maxes out....



To be honest though, the code here is really irrelevant I think. When I replaced the document values with a blank string "" - The problem was still the same










share|improve this question
















I'm using Couchbase, with .net - Latest CouchbaseNetClient.



Couchbase Serer 4.6.2 Enterprise running.



I have an app where It performs 5k/ops.



The problem I am facing - is as soon as I start this app - then the CPU usage jumps to 100% and Disk Usage jumps to 100% also (as noted on the ctrl+alt+del ->Performance->Disk 0 (C:)



I have tested across a few PC.s - All are SSD (Samsung 850 PRO) and a dell M.2. Both have excellent r/w capabilities which I would have thought would swallow 5k ops.



The machine with the 850 PRO is a 6c/12t 980x CPU.
The machine with the m2.dell is a 4c/8t i76700hq.



I'm using a standard Upsert as per couchbase doco's. When I comment out this line, CPU goes right down to expected levels.



Is there some form of tweaking/performance/indexing mechanism I am missing entirely here??



Regards,



Chud



EDIT #1



So I uninstalled Couchbase on the Dell XPS and reinstalled it as memcached server only. Funny thing is, the CPU goes to 100% and now disk stays at 0% (expected of course). I have commented out the lines that actually do the Upsert, and CPU usage remains virtually unchanged to what it was before I started the app, even though the app itself is still doing a fair bit of crunching, it's just not entering the data into the (now) memcached database.



How can the upsert be so so so expensive? It's only a small Json file...



EDIT #2



So here is some samples that may get you going. I have changed the models around a some what to keep confidentiality in what I am doing, so it is "untested" so to speak. So if there are bugs in this I apologise, I literally just hacked it together.



Json Sample Below - I guess put this in a loop and let it rain, see if you can produce the same load.



{"Position":10,"Type":0,"Type2":5,"Value":0.0,"Type3":1,"Location":"Bla","Type":"Ex","Id":"11111"}

public class RootObject
{
public int Position { get; set; }
public MyFirstType Type { get; set; }
public MySecondType Type2 { get; set; }
public double Value { get; set; }
public MyThirdType Type3 { get; set; }
public string Location { get; set; }
public string Id { get; set; }
}

public enum MyFirstType
{
A,
B
}

public enum MySecondType
{
C,
D,
E,
F,
G,
H
}

public enum MyThirdType
{
I,
J
}
public class CouchBaseInstanceTest
{
private Cluster CouchClient;
private IBucket Bucket;

public CouchBaseInstanceTest()
{
CouchClient = new Cluster(new ClientConfiguration
{
Servers = new List<Uri> {new Uri("http://localhost:8091")}

});
// I think CB v 5.0 and over, or 5.5 needs auth against the bucket
//CouchClient.Authenticate("admin", "password");
Bucket = CouchClient.OpenBucket("default");
}
public bool Upsert(string key, dynamic value, double expiry)
{
var Document = new Document<dynamic>()
{
Id = key,
Content = value,
Expiry = TimeSpan.FromHours(expiry).ToTtl()
};
return Bucket.Upsert(Document).Success;
}
}


So as soon as that Bucket.Upsert is hit - the CPU/Disk maxes out.... or in just the memcached config....the CPU maxes out....



To be honest though, the code here is really irrelevant I think. When I replaced the document values with a blank string "" - The problem was still the same







c# .net couchbase






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 20 '18 at 23:17







The_Chud

















asked Nov 19 '18 at 6:17









The_ChudThe_Chud

195110




195110













  • Sounds like you should ask this on github or where ever this is located.

    – TheGeneral
    Nov 19 '18 at 6:28











  • Could you post the minimum code to recreate this? It's possible that you're creating too many connections or something like that.

    – Matthew Groves
    Nov 20 '18 at 15:01











  • Hi Matthew, I actually use a DI Container to establish the connection. This is only done once on program startup, and then the only thing called is an Upsert Function. I have even tried to use the Upsert(Dictionary<string,dynamic> "bulk" type function, but it makes no difference.

    – The_Chud
    Nov 20 '18 at 22:31



















  • Sounds like you should ask this on github or where ever this is located.

    – TheGeneral
    Nov 19 '18 at 6:28











  • Could you post the minimum code to recreate this? It's possible that you're creating too many connections or something like that.

    – Matthew Groves
    Nov 20 '18 at 15:01











  • Hi Matthew, I actually use a DI Container to establish the connection. This is only done once on program startup, and then the only thing called is an Upsert Function. I have even tried to use the Upsert(Dictionary<string,dynamic> "bulk" type function, but it makes no difference.

    – The_Chud
    Nov 20 '18 at 22:31

















Sounds like you should ask this on github or where ever this is located.

– TheGeneral
Nov 19 '18 at 6:28





Sounds like you should ask this on github or where ever this is located.

– TheGeneral
Nov 19 '18 at 6:28













Could you post the minimum code to recreate this? It's possible that you're creating too many connections or something like that.

– Matthew Groves
Nov 20 '18 at 15:01





Could you post the minimum code to recreate this? It's possible that you're creating too many connections or something like that.

– Matthew Groves
Nov 20 '18 at 15:01













Hi Matthew, I actually use a DI Container to establish the connection. This is only done once on program startup, and then the only thing called is an Upsert Function. I have even tried to use the Upsert(Dictionary<string,dynamic> "bulk" type function, but it makes no difference.

– The_Chud
Nov 20 '18 at 22:31





Hi Matthew, I actually use a DI Container to establish the connection. This is only done once on program startup, and then the only thing called is an Upsert Function. I have even tried to use the Upsert(Dictionary<string,dynamic> "bulk" type function, but it makes no difference.

– The_Chud
Nov 20 '18 at 22:31












0






active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53369249%2fcouchbase-disk-usage%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes
















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53369249%2fcouchbase-disk-usage%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Biblatex bibliography style without URLs when DOI exists (in Overleaf with Zotero bibliography)

ComboBox Display Member on multiple fields

Is it possible to collect Nectar points via Trainline?