Clustering ip-addresses on domain names
I have an ip-network which is basically a list of sequential ip-addresses. From this list I want to cluster ranges of ip-addresses into independent entities. I want to give each IP in the range a set of properties like time to live, nameservers and domain names associated with it.
I then want to determine the distance between each IP-address and its neighbors and start clustering based on shortest distance.
My question lies in the distance function. TTL is a number so that should not be a problem. Nameservers and domain names are strings however, how would you represent those as numbers in a vector?
Basically if 2 IP-addresses have the same nameserver or very similar domain names (equal 2LD) you want them to have a smaller distance. I've looked into something like word2vec but can't really find a useful implementation.
python ip ip-address word2vec hierarchical-clustering
asked Nov 21 '18 at 16:33
mBomBo
407
add a comment |
I have an ip-network which is basically a list of sequential ip-addresses. From this list I want to cluster ranges of ip-addresses into independent entities. I want to give each IP in the range a set of properties like time to live, nameservers and domain names associated with it.
I then want to determine the distance between each IP-address and its neighbors and start clustering based on shortest distance.
My question lies in the distance function. TTL is a number so that should not be a problem. Nameservers and domain names are strings however, how would you represent those as numbers in a vector?
Basically if 2 IP-addresses have the same nameserver or very similar domain names (equal 2LD) you want them to have a smaller distance. I've looked into something like word2vec but can't really find a useful implementation.
python ip ip-address word2vec hierarchical-clustering
asked Nov 21 '18 at 16:33
mBomBo
407
add a comment |
I have an ip-network which is basically a list of sequential ip-addresses. From this list I want to cluster ranges of ip-addresses into independent entities. I want to give each IP in the range a set of properties like time to live, nameservers and domain names associated with it.
I then want to determine the distance between each IP-address and its neighbors and start clustering based on shortest distance.
My question lies in the distance function. TTL is a number so that should not be a problem. Nameservers and domain names are strings however, how would you represent those as numbers in a vector?
Basically if 2 IP-addresses have the same nameserver or very similar domain names (equal 2LD) you want them to have a smaller distance. I've looked into something like word2vec but can't really find a useful implementation.
python ip ip-address word2vec hierarchical-clustering
asked Nov 21 '18 at 16:33
mBomBo
407
I have an ip-network which is basically a list of sequential ip-addresses. From this list I want to cluster ranges of ip-addresses into independent entities. I want to give each IP in the range a set of properties like time to live, nameservers and domain names associated with it.
I then want to determine the distance between each IP-address and its neighbors and start clustering based on shortest distance.
My question lies in the distance function. TTL is a number so that should not be a problem. Nameservers and domain names are strings however, how would you represent those as numbers in a vector?
Basically if 2 IP-addresses have the same nameserver or very similar domain names (equal 2LD) you want them to have a smaller distance. I've looked into something like word2vec but can't really find a useful implementation.
python ip ip-address word2vec hierarchical-clustering
python ip ip-address word2vec hierarchical-clustering
asked Nov 21 '18 at 16:33
mBomBo
407
asked Nov 21 '18 at 16:33
mBomBo
407
asked Nov 21 '18 at 16:33
mBomBo
407
asked Nov 21 '18 at 16:33
mBomBo
407
asked Nov 21 '18 at 16:33
mBomBo
407
407
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
I would try using difflib like this.
from difflib import SequenceMatcher
def similarity(a, b):
return SequenceMatcher(None, a, b).ratio()
Then you can call the function against each set of names to get a similarity score and group them based on that.
similarity("server1","server1")
1.0
similarity("Server1","Server2")
0.8571428571428571
similarity("foo","bar")
0.0
answered Nov 21 '18 at 16:52
Seth WahleSeth Wahle
103111
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53416627%2fclustering-ip-addresses-on-domain-names%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I would try using difflib like this.
from difflib import SequenceMatcher
def similarity(a, b):
return SequenceMatcher(None, a, b).ratio()
Then you can call the function against each set of names to get a similarity score and group them based on that.
similarity("server1","server1")
1.0
similarity("Server1","Server2")
0.8571428571428571
similarity("foo","bar")
0.0
answered Nov 21 '18 at 16:52
Seth WahleSeth Wahle
103111
add a comment |
I would try using difflib like this.
from difflib import SequenceMatcher
def similarity(a, b):
return SequenceMatcher(None, a, b).ratio()
Then you can call the function against each set of names to get a similarity score and group them based on that.
similarity("server1","server1")
1.0
similarity("Server1","Server2")
0.8571428571428571
similarity("foo","bar")
0.0
answered Nov 21 '18 at 16:52
Seth WahleSeth Wahle
103111
add a comment |
I would try using difflib like this.
from difflib import SequenceMatcher
def similarity(a, b):
return SequenceMatcher(None, a, b).ratio()
Then you can call the function against each set of names to get a similarity score and group them based on that.
similarity("server1","server1")
1.0
similarity("Server1","Server2")
0.8571428571428571
similarity("foo","bar")
0.0
answered Nov 21 '18 at 16:52
Seth WahleSeth Wahle
103111
I would try using difflib like this.
from difflib import SequenceMatcher
def similarity(a, b):
return SequenceMatcher(None, a, b).ratio()
Then you can call the function against each set of names to get a similarity score and group them based on that.
similarity("server1","server1")
1.0
similarity("Server1","Server2")
0.8571428571428571
similarity("foo","bar")
0.0
answered Nov 21 '18 at 16:52
Seth WahleSeth Wahle
103111
answered Nov 21 '18 at 16:52
Seth WahleSeth Wahle
103111
answered Nov 21 '18 at 16:52
Seth WahleSeth Wahle
103111
answered Nov 21 '18 at 16:52
Seth WahleSeth Wahle
103111
103111
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53416627%2fclustering-ip-addresses-on-domain-names%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
