Jump to content

Cluster Computing


Recommended Posts

So i thought id toy with a few old computers i can get at a surplus place, while learning linux in college, thought id make some self learning before i get into the classes...

 

But i...dont have a clue how to do this.lol I cant really find anything on HOW to set things up... other than the basic network setup. I know how to do this... and that each CPU would need to different for it to work correctly.

 

For setup... I dunno..

 

Could Knoppix/Red Hat work?

 

Or would i need to run your Command Line version of linux to make this all work- makes more sense since there wouldnt be any monitors on the nodes in all.

 

Someone help a kid with an personal challenge to himself

Link to post
Share on other sites
  • Replies 56
  • Created
  • Last Reply

Top Posters In This Topic

So i thought id toy with a few old computers i can get at a surplus place, while learning linux in college, thought id make some self learning before i get into the classes...

 

But i...dont have a clue how to do this.lol I cant really find anything on HOW to set things up... other than the basic network setup. I know how to do this... and that each CPU would need to different for it to work correctly.

 

For setup... I dunno..

 

Could Knoppix/Red Hat work?

 

Or would i need to run your Command Line version of linux to make this all work- makes more sense since there wouldnt be any monitors on the nodes in all.

 

Someone help a kid with an personal challenge to himself

 

There is no golden button which does it for you, it is a long and comlex process requiring alot of knowledge.

 

Only way to do it is read, read, read.

 

But as a pointer, Debian is apparently the way to go.

Link to post
Share on other sites

ahhh thanks very much Bruce, that is very much what i was looking for.

 

Oh yeah i figure its not going to be one of those.... point...click.. run.. kinda operations...i need a project to give me something do to and help advance my knowledge of computers, networking, and abilties of a computer. I want to learn! I find myself wondering what i can do with myself and I was just google'ing things and i ran across this and peaked interest.

 

There is always a guide to something ;)

 

Anyone here try clustering?

Link to post
Share on other sites

ahhh thanks very much Bruce, that is very much what i was looking for.

 

Oh yeah i figure its not going to be one of those.... point...click.. run.. kinda operations...i need a project to give me something do to and help advance my knowledge of computers, networking, and abilties of a computer. I want to learn! I find myself wondering what i can do with myself and I was just google'ing things and i ran across this and peaked interest.

 

There is always a guide to something ;)

 

Anyone here try clustering?

 

XxenxX knows of a package for Debian and I think he has plans to add a personal cluster soon.

Link to post
Share on other sites

I have the packages for Debian Sarge if you want them. It's pretty easy to do actually.

 

1) dpkg -i the kernel package on each machine.

2) Reboot said machines into their new kernels.

3) Run omdiscd on each machine (you can make a boot script if you want).

4) Install the client programs on the machine you want to be the master.

 

Doesn't really take any configuration. The only reason I haven't set this up for more than testing is that I don't have (and can't find) the kernel source for the kernel package. Most of my machines have NVidia graphics cards, and you need the kernel source installed to compile the driver for them... so for the time being, it just won't work. Eventually I'll drag myself to patch a more recent kernel with the OpenMosix stuff, and get eveything going then...

 

If you want to try a cluster without installing anything, try ClusterKnoppix.

Edited by xXenXx
Link to post
Share on other sites

Oo kool... cluster knoppix looks easier.

 

 

Ok what you said flew over my head on the second part. lol

 

from what i got, your computer have external vid card- rather than on board..and your having problems with drivers for it all to run?

 

First part. I take it after i put the OS on the HD, i type in those commands... or as i install it? Also i can put Knoppix on the HD's right... thought i could.

 

Id like the OS if you could spare it... just save it to computer then burn it to CD right? How most go.. :P

Link to post
Share on other sites

You might want to learn the basics of linux before you try anything...

 

As for my dilema, I like having 3d acceleration. I can't have that unless I install the drivers for my video card. The drivers require me to build a module, and said module requires the kernel source. I don't have the kernel source, that being the problem.

 

No you cannot install ClusterKnoppix to the harddrive, though you can make a permanent home partition on it, or save your settings.

 

Install the operating system, then install OpenMosix. You can get Debian from their website (it's in my signature).

Link to post
Share on other sites

well yeah.. planning ahead :D

 

OK install Debian GNU Linux...then Install OpenMosix- where?

 

Then i repeat n times...

 

OR i just burn off n disks of ClusterKnoppix, setup whatever, and then im nearly done.

 

Im close...or not. lmao

Link to post
Share on other sites

very slightly...

 

Knoppix and RedHat-latest release, and about the same time under command line for it.

 

Like i said im going into class and im just going to self learn it before i go into the class.

 

Asking questions = learning :)

Link to post
Share on other sites

Debian is excellent. But if you already have experience with Redhat, you may consider 2 free versions of RHEL-AS that both have clustered computing capabilites. They are below with links.

 

Scientific Linux in particular is used at major scientific centers like FermiLab and Cern. Both of those places run huge computer "farms" on Scientific Linux to analyze 100's of terrabytes of data generated by their particle accelerators/detectors.

 

If you install Scientific Linux, when you get to the software selections, scroll down to the bottom and there is an "everything" choice. While I'm sure you probably don't want everything, this is nice because it will install the RHEL manuals also. They will be in a selection in your main menu. One can learn a lot of basics reading these manuals.

 

Scientific Linux

 

https://www.scientificlinux.org/

 

ftp://ftp.scientificlinux.org/linux/scientific/44/iso/

 

Scientific Linux Public Mirrors

 

 

CentOS

 

https://www.centos.org/

 

CentOS Public Mirrors

 

 

RHEL-AS

 

https://www.redhat.com/

Edited by John 44
Link to post
Share on other sites

hmm that is interesting... ill reply once i get the equipment and nodes all setup once i get them.

 

Either you have AIM or MSN i could chat freely with?

Apple_jack2004#yahoo.com- MSN

AndewJames2004- AIM

Link to post
Share on other sites

Willing but not able. I type too slowly for chat. But more importantly, my experience with clustered computing amounts to 3 machines for about 2 days total; and that was a couple of years ago.

 

I did it just to see if I could. I don't have anything to compute that needs that kinda power. Nor do I have a number of identical machines that would be optimal tho not required if I remember.

Link to post
Share on other sites

But ya got it to work :P

 

PM's will work?

 

They sure will. And so would email and telephone. :P But none of those options can make up for my lack of recent experience.

 

If your serious about this and want my input, I'll have to do again myself 1st. That should not take more than week I should think. :lol: I still have an 8 port gigabit switch and can probably cobble together 2 nearly identical machines.

 

I'm retired and have nothing but (too much) time. Lemme know.

 

Debian with xXenXx's help may do all you need though.

Link to post
Share on other sites

There's a lotta good reading there. Step by Step is a good starting place I should think. Fair warning though: Linux grows and changes fast. The published documentation for every distro is always behind. So, don't be surprised to find a particular detail that is not current.

 

Posted Image

Link to post
Share on other sites

My method is two commands per machine.

 

No, you don't need identical hardware to run a cluster. All of the computers just have to be the same architecture, and being running the same kernel.

 

I type fast. Either tom9729 on aim, or [email protected] on msn. I'll help you with pretty much anything if I'm not busy, but I'm not 24 hour tech support (unless you want to pay me ;)).

 

Also, Debian has far more software than any of the RedHat(s).

 

An nice alternative to running a cluster, is having several machines set up as headless slaves (interesting sound of that eh?), with SSH -X servers running on them. You can then run something like an internet browser off one machine, and have it show up on yours (memory footprint of the browser stays on the remote machine. The one problem is that you have to do everything manually.

 

Just to tell you, a cluster of only 3-5 machines isn't a lot of processing power.

 

 

Sorry for that extremely broken up post, I'm exhausted. I shouldn't be allowed to post on here without a certain amount of sleep. My posts certainly suffer from it...

Link to post
Share on other sites

question tho... how much processing power would say 4 PII and a amd athlon xp 3200+ do?

I have no idea really but I will venture a WAG anyway. :lol:

 

Depends on what you're computing. Consider 2 cases. My WAGGED numbers are meaningless quantitatively because I don't know. But,.....

 

Case 1:

 

You are computing something that takes 1 hr on that xp 3200+ alone. Now, set up your cluster and do it again. It might take 30 minutes or 90 minutes depending on the cluster hardwar/software efficiency.

 

Case 2:

 

You are computing something that takes 30 days on that xp 3200+ alone. The same cluster may cut that to 15 days.

 

In other words, the bigger the job, the more helpful the extra machines can be. The smaller the job, the less help they may be due to the "overhead".

 

Reminds me of the old joke. How many [insert your favorite group here]s does it take to screw in a lightbulb? Answer: 10; 1 to hold the lightbulb and 9 to turn the ladder around.

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...