Data mining, Data analysis, Data base... Chalanachithram.com | Topics | Search
Hide Clipart | Log Out | Register | Edit Profile

Last 30 mins | 1 | 2 | 4 hours     Last 1 | 7 Days

Chalanachithram.com DB » TF Industry related » Archive through August 01, 2008 » Data mining, Data analysis, Data base experts!! « Previous Next »

Author Message
Top of pagePrevious messageNext messageBottom of page Link to this message

Webbugger
Junior Artist
Username: Webbugger

Post Number: 433
Registered: 03-2008
Posted From: 164.54.210.16

Rating: N/A
Votes: 0 (Vote!)

Posted on Thursday, July 31, 2008 - 12:32 pm:   Insert Quote Edit PostDelete PostPrint Post   Move Post (Moderator/Admin Only)Ban Poster IP (Moderator/Admin only)

DB can't handle these huge files... file storage much easier to manage...you can only store the paths in DB...

Use efficient SAN (Storage Area Network) to store & access the data. i don't know on windows...in linux we use GFS (Global file system), here in my lab images are generated each of 30MB , around 200GB per day)
http://www.andhrastage.com
http://www.indiastage.com
Top of pagePrevious messageNext messageBottom of page Link to this message

Pavala
Side Hero
Username: Pavala

Post Number: 2175
Registered: 02-2008
Posted From: 193.47.71.253

Rating: N/A
Votes: 0 (Vote!)

Posted on Thursday, July 31, 2008 - 11:29 am:   Insert Quote Edit PostDelete PostPrint Post   Move Post (Moderator/Admin Only)Ban Poster IP (Moderator/Admin only)


Maverick:

CFD analysis aa enti



Ledhu,
Wind Turbine simulator software...120 channels...50 Hz sampling frequency and 10 minutes data...this is file size....

Oka one month collect ayina files ni process cheyyaali plus ee files ni windows network drive nundi access cheyyaali...chirak dobbs.....suggestions ivvu..
Yevadu boora voodhithey Pamulannee bussumani puttalo nundi bhayataku vasthaayo, vaade Pavala
Top of pagePrevious messageNext messageBottom of page Link to this message

Maverick
Side Hero
Username: Maverick

Post Number: 3661
Registered: 01-2008
Posted From: 192.146.101.24

Rating: N/A
Votes: 0 (Vote!)

Posted on Thursday, July 31, 2008 - 11:26 am:   Insert Quote Edit PostDelete PostPrint Post   Move Post (Moderator/Admin Only)Ban Poster IP (Moderator/Admin only)


Savage:

rest god grace




rest prof eignatha
Top of pagePrevious messageNext messageBottom of page Link to this message

Savage
Side Hero
Username: Savage

Post Number: 2348
Registered: 02-2008
Posted From: 148.129.71.53

Rating: N/A
Votes: 0 (Vote!)

Posted on Thursday, July 31, 2008 - 11:25 am:   Insert Quote Edit PostDelete PostPrint Post   Move Post (Moderator/Admin Only)Ban Poster IP (Moderator/Admin only)


Pavala:


Hater ni research anta DB members to seyistunnava, blood petti seyyi rest god grace
Top of pagePrevious messageNext messageBottom of page Link to this message

Maverick
Side Hero
Username: Maverick

Post Number: 3660
Registered: 01-2008
Posted From: 192.146.101.24

Rating: N/A
Votes: 0 (Vote!)

Posted on Thursday, July 31, 2008 - 11:25 am:   Insert Quote Edit PostDelete PostPrint Post   Move Post (Moderator/Admin Only)Ban Poster IP (Moderator/Admin only)


Pavala:




CFD analysis aa enti
Top of pagePrevious messageNext messageBottom of page Link to this message

Pavala
Side Hero
Username: Pavala

Post Number: 2173
Registered: 02-2008
Posted From: 193.47.71.253

Rating: N/A
Votes: 0 (Vote!)

Posted on Thursday, July 31, 2008 - 11:24 am:   Insert Quote Edit PostDelete PostPrint Post   Move Post (Moderator/Admin Only)Ban Poster IP (Moderator/Admin only)


Proline:



Annai
File writing is not an issue. We get this data from some machine externally which we post-process offline for analysis using Matlab. All the data is stored on a network drive in windows :-(

But prathee saari matlab lo data invoke chesthuntey chiruguthunnaadhi. Network drive lo fast ga access chesey utility ledhaa ? ledhantey remote log in chesi dobbinchukovadam better aa ?
Yevadu boora voodhithey Pamulannee bussumani puttalo nundi bhayataku vasthaayo, vaade Pavala
Top of pagePrevious messageNext messageBottom of page Link to this message

Proline
Junior Artist
Username: Proline

Post Number: 59
Registered: 06-2008
Posted From: 170.22.76.10

Rating: N/A
Votes: 0 (Vote!)

Posted on Thursday, July 31, 2008 - 11:15 am:   Insert Quote Edit PostDelete PostPrint Post   Move Post (Moderator/Admin Only)Ban Poster IP (Moderator/Admin only)

pavala: idigo naa cent...

file writing is always slower than directly storing the data inside DB or caching systems.

find out if your sensor systems can integrate with DB (or DB can integrate with ur sensor systems). may need to write a driver/adapter to integrate ur sensor systmes and database.
Top of pagePrevious messageNext messageBottom of page Link to this message

Pavala
Side Hero
Username: Pavala

Post Number: 2170
Registered: 02-2008
Posted From: 193.47.71.253

Rating: N/A
Votes: 0 (Vote!)

Posted on Thursday, July 31, 2008 - 11:06 am:   Insert Quote Edit PostDelete PostPrint Post   Move Post (Moderator/Admin Only)Ban Poster IP (Moderator/Admin only)

Guys

I have a question.

If a machine keeps generating data (signals from around 100 sensors) into files at certain time periods, which is the optimal way of storing them ? The data is in binary format. After certain time, the statistical analysis is carried out on the whole data.

Did anyone design such systems before? We are using a crude way of storing the files in windows file system and then use Pascal based programmes for generating statistical data.

Any improvement suggestions ? Which software/tools are the best ?

Thanks in advance
Yevadu boora voodhithey Pamulannee bussumani puttalo nundi bhayataku vasthaayo, vaade Pavala

Add Your Message Here
Post:
Bold text Italics Underline Create a hyperlink Insert a clipart image HASH(0x9e7e648){Movie Clipart}
Show / Hide regular icons selection options

Click on following links to open cliparts by Alphabetical Order

 A   B   C   D   E   F   G   H   I   J   K   L   M  

 N   O   P   Q   R   S   T   U   V   W   X   Y   Z  

Show / Hide Filmy icons selection options

Click on following links to open cliparts by Alphabetical Order

 A   B   C   D   E   F   G   H   I   J   K   L   M  

 N   O   P   Q   R   S   T   U   V   W   X   Y   Z  


Username: Posting Information:
This is a public posting area. Enter your username and password if you have an account. Otherwise, enter your full name as your username and leave the password blank. Your e-mail address is optional.
Password:
E-mail:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: