Topics | Search Log Out | Register | Edit Profile
Hide Clipart | Banned/Unbanned User Log | Moderator Login History | Thread Delete/Move Log | Last 30 mins | 1 | 2
Big Data : Apache Kafka

Chalanachithram.com DB » New TF Industry Related » Archive through September 30, 2016 » Big Data : Apache Kafka « Previous Next »
Author Message
 

Gandhiguevara
Legend
Username: Gandhiguevara

Post Number: 61756
Registered: 10-2009
Posted From: 67.191.221.7

Rating: N/A
Votes: 0 (Vote!)

Posted on Thursday, September 29, 2016 - 08:18 pm:       


Myselfme:

but moving to Cloudera distribution flafka or kafka-kafka


kafka-kafka enti? multi data centrer set up aaa?
 

Jeffa
Junior Artist
Username: Jeffa

Post Number: 751
Registered: 10-2012
Posted From: 50.185.122.221

Rating: N/A
Votes: 0 (Vote!)

Posted on Thursday, September 29, 2016 - 12:43 am:       


Myselfme:

1. Apache Nifi, Talend have UI which are used for data ingestion, all the dashboard softwares like Tableau have ability to query HDFS.

2. Big data transactions can be possible with http://www.splicemachine.com/

3. terminalogy: Machnine Learning, deep learning, streaming




Erra bassu kitiki lonunchi erusenakkayal konukkunnattu enti ee comparison thammi? Jambu em spetunnadu, nuvvu em septunnav? Assalu dinner ayindaa leda? :-)
Nifi ki vunna UI enti? Jambu raja antunna UI enti?
 

Myselfme
Junior Artist
Username: Myselfme

Post Number: 322
Registered: 04-2011
Posted From: 73.22.114.102

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 11:49 pm:       

Jambalhaart:

1. Apache Nifi, Talend have UI which are used for data ingestion, all the dashboard softwares like Tableau have ability to query HDFS.

2. Big data transactions can be possible with http://www.splicemachine.com/

3. terminalogy: Machnine Learning, deep learning, streaming
 

Siloan
Legend
Username: Siloan

Post Number: 48037
Registered: 03-2008
Posted From: 132.174.137.234

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 11:47 pm:       


Jambalahaart_raja:

Big Data is not directly related to real-time transactions DB.



very soon IBM is going to
 

Jambalahaart_raja
Side Hero
Username: Jambalahaart_raja

Post Number: 6950
Registered: 07-2008
Posted From: 24.45.107.249

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 11:28 pm:       

1. Big Data has no UI involvement.
2. Big Data is not directly related to real-time transactions DB.
3. Big Data is Data-Mining, simply put, processing large chunks of data, breaking them down into simpler, readily accessible, codependent, inter-related smaller chunks of data for more processing eventually. This has to be real-time, to facilitate all the other real-time transactions based systems.
Inthenaa.. inkemaina cheppadaluchukunnara??
"Chill Bro.
I told you to let it go!!"
- The Budhha.
 

Jeffa
Junior Artist
Username: Jeffa

Post Number: 750
Registered: 10-2012
Posted From: 204.44.10.4

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 07:28 pm:       


Dma:

active mq consumers distributed env lo work kaavu.
kafka can be used in distributed env with other big data echo system.



in other words, for all BigD(ata) systems, everything should be distributed system only.
 

Jeffa
Junior Artist
Username: Jeffa

Post Number: 749
Registered: 10-2012
Posted From: 204.44.10.4

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 07:26 pm:       

Recently I worked on Data Torrents (Apache Apex Framework) and did 2 POCs and one project. After that I didnt like Spark :-)
 

Myselfme
Junior Artist
Username: Myselfme

Post Number: 321
Registered: 04-2011
Posted From: 73.22.114.102

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 06:18 pm:       

We are using Kafka through Ambari, I do not know the history in my company, some how kafka cluster have all the files in HDP directory, so understood somehow they are usng from hortonworks, admins won't reveal deatils

but moving to Cloudera distribution flafka or kafka-kafka
 

Dma
Hero
Username: Dma

Post Number: 17716
Registered: 11-2009
Posted From: 172.56.17.4

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 06:07 pm:       


Siloan:

active mq ki kafka ki teda enti ?




same concept.

active mq consumers distributed env lo work kaavu.
kafka can be used in distributed env with other big data echo system.
Jai Andhra!! Jai Jai Andhra!!!
 

Dma
Hero
Username: Dma

Post Number: 17715
Registered: 11-2009
Posted From: 172.56.17.4

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 06:05 pm:       

tes.

kafka, storm, mongo, flume
Jai Andhra!! Jai Jai Andhra!!!
 

Emc2
Legend
Username: Emc2

Post Number: 50883
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 04:23 pm:       


Last_avataar:




now i got it, when i had fraudulent transaction on my CC ,you are the one who is processing and alerting me ani ardham ayindi,

I am assuming you guys are analyzing these on real time..

thanks
 

Maverick
Legend
Username: Maverick

Post Number: 62118
Registered: 01-2008
Posted From: 108.171.129.189

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 04:17 pm:       


Emc2:

pretty much minutes lo millions lo dealings chesthunnanu



Happy Vizag
 

Last_avataar
Side Hero
Username: Last_avataar

Post Number: 7419
Registered: 09-2012
Posted From: 12.10.219.224

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 04:16 pm:       


Emc2:




Banking and Financial Sir
 

Emc2
Legend
Username: Emc2

Post Number: 50882
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 04:13 pm:       


Last_avataar:




no i am not asking for case study, just tell the industry if passible
 

Last_avataar
Side Hero
Username: Last_avataar

Post Number: 7417
Registered: 09-2012
Posted From: 12.10.219.224

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 04:10 pm:       


Emc2:




sorry i cannot discuss that case study in any forum.
cheppenuga meeku 3 days lo ichedi , few hrs lo ichenu anukondi, i can do better bargain with you
 

Siloan
Legend
Username: Siloan

Post Number: 48031
Registered: 03-2008
Posted From: 132.174.20.41

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 04:09 pm:       


Emc2:

ellehe, avtive mq kuda open source,



active mq ki kafka ki teda enti ?
 

Emc2
Legend
Username: Emc2

Post Number: 50881
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 04:01 pm:       


Last_avataar:




ante raju garu memu kuda bpm's and multi threaded Q's ,jboss data grids use chesi pretty much minutes lo millions of transactions chesthunnamu

mee case study or purpose enti?
 

Emc2
Legend
Username: Emc2

Post Number: 50880
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 03:59 pm:       


Siloan:




ellehe, avtive mq kuda open source,
 

Last_avataar
Side Hero
Username: Last_avataar

Post Number: 7416
Registered: 09-2012
Posted From: 12.10.219.224

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 03:58 pm:       


Emc2:




In Any Industry where we process transactions, that can reduce the time from days to with in day or hrs , Business value increases and better bargains can happen with all the stakeholders and actor
 

Siloan
Legend
Username: Siloan

Post Number: 48030
Registered: 03-2008
Posted From: 132.174.20.41

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 03:56 pm:       


Emc2:

etl's and other stuff ki active mq or tibco lantivi kuda saripothayi




open source
 

Emc2
Legend
Username: Emc2

Post Number: 50879
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 03:54 pm:       


Siloan:




etl's and other stuff ki active mq or tibco lantivi kuda saripothayi

unless there is a need to integrate and process the big data kafka avasaram ledu

of course regular processing ki kuda use cheyochu, but trying to know the real need
 

Siloan
Legend
Username: Siloan

Post Number: 48029
Registered: 03-2008
Posted From: 132.174.20.41

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 03:49 pm:       


Emc2:

ante antha high intensity message broker using ki reasons enti




edm qvechhan appa ...messaging thru etl movement avoid cheyochhu...SLA's ki shaana help avuddi
 

Emc2
Legend
Username: Emc2

Post Number: 50878
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 03:37 pm:       


Last_avataar:




what kind of industry you are working

ante antha high intensity message broker using ki reasons enti

just give high level, details not required for privacy
 

Last_avataar
Side Hero
Username: Last_avataar

Post Number: 7401
Registered: 09-2012
Posted From: 12.10.219.223

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 02:04 pm:       


Gandhiguevara:




Problem emitante , info and data security is 200% tight, not even a byte can be copied from office systems, even it is personal.

Camel Client is used for slicing the file, putting messages to Kafka Topics
Kafaka topic to send MESSAGE to Consumer running on iGNITE server
Consumer to read message and handover to Ignite process
Ignite to process the message using (pre-loaded ) in Memory Data
if Data unavailable in Memory get it from DB
once processed , Ignite to response to Consumer
Consumer to keep it back in Response Topic of Kafka
Assembler to read all Kafka responses to assemble related messages to file
Script to send the file back to Secure File system
This goes on like cycle.
 

Gandhiguevara
Legend
Username: Gandhiguevara

Post Number: 61753
Registered: 10-2009
Posted From: 67.191.221.7

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:57 pm:       


Last_avataar:

Apache camel for file streaming, Kakfa for splitting file to Messaging , Ignite for In Memory computing, and sending the processed messages back in the same route assemble as file sent to upstream system.

on open JDK 1.8

Point of Arrival is get message, process, send message directly back to upstream


interesting...I have a similar use case...emanna bommal vunnayaa architecture vi rakesh.ccdb@gmail.com
 

Last_avataar
Side Hero
Username: Last_avataar

Post Number: 7397
Registered: 09-2012
Posted From: 12.10.219.223

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:49 pm:       


Saarang:




It takes one project to get you back on the track, need to slog
Chiraku denks to learn new set of technologies each year
But taffadu
 

Saarang
Hero
Username: Saarang

Post Number: 13773
Registered: 06-2012
Posted From: 206.29.176.75

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:46 pm:       

Ee perlu anni naaku Yamaha 1234 type lo anipistunnai enti...baaga outdated ayyipoyaana

 

Last_avataar
Side Hero
Username: Last_avataar

Post Number: 7395
Registered: 09-2012
Posted From: 12.10.219.221

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:45 pm:       


Gandhiguevara:




Apache camel for file streaming, Kakfa for splitting file to Messaging , Ignite for In Memory computing, and sending the processed messages back in the same route assemble as file sent to upstream system.

on open JDK 1.8

Point of Arrival is get message, process, send message directly back to upstream
 

Emc2
Legend
Username: Emc2

Post Number: 50876
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:43 pm:       


Gandhiguevara:




no rajaa, serious , but i don't want to travel, adokkate samasya

at present ayithe hadoop, pig ,greenplum,aws, ilantivi koncham using on and off..

but billing kuda antha goppa ga anipinchaledu, so thinking
 

Gandhiguevara
Legend
Username: Gandhiguevara

Post Number: 61751
Registered: 10-2009
Posted From: 67.191.221.7

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:41 pm:       


Last_avataar:

Camel


or Camus?
 

Last_avataar
Side Hero
Username: Last_avataar

Post Number: 7393
Registered: 09-2012
Posted From: 12.10.219.221

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:40 pm:       


Gandhiguevara:

evaranna vaaduthunnaaraa?




yes we are working on Camel, Apache kafka, Apache ignite on In Memory computing Project
 

Gandhiguevara
Legend
Username: Gandhiguevara

Post Number: 61750
Registered: 10-2009
Posted From: 67.191.221.7

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:40 pm:       


Emc2:

Flink - Apache Flink has an integration with Kafka
Samza - A YARN-based stream processing framework.

job ichesthara, emundi unkl dantlo, api and stating points setup chusthe get started ayipothadi


koncham line loki vasthaav....serious aithe I can guide...just leg pulling and oil pulling aithe I can entertain too
 

Emc2
Legend
Username: Emc2

Post Number: 50875
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:33 pm:       


Gandhiguevara:

samza flink flafka




unkl mottam eco system 2 mins lo batti pattesa

Flink - Apache Flink has an integration with Kafka
Samza - A YARN-based stream processing framework.

job ichesthara, emundi unkl dantlo, api and stating points setup chusthe get started ayipothadi

next koncham time teesukuni advanced techniqies scripting integrations,

 

Gandhiguevara
Legend
Username: Gandhiguevara

Post Number: 61748
Registered: 10-2009
Posted From: 67.191.221.7

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:33 pm:       


Siloan:

check rakesh


ok
 

Siloan
Legend
Username: Siloan

Post Number: 48027
Registered: 03-2008
Posted From: 132.174.20.41

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:31 pm:       


Gandhiguevara:



check rakesh
 

Gandhiguevara
Legend
Username: Gandhiguevara

Post Number: 61746
Registered: 10-2009
Posted From: 67.191.221.7

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:28 pm:       


Emc2:

40 waste ayithe 39 iyyi chalu, kani oka sample resume, interview lekunda 39 ki oka job in kafka ipinchu chalu


mundu nuvvu konni words nerchuko kafka streaming logs samza flink flafka ani...neekedho telsu anukuni $54 varaku ichesthaaru
 

Gandhiguevara
Legend
Username: Gandhiguevara

Post Number: 61745
Registered: 10-2009
Posted From: 67.191.221.7

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:27 pm:       


Netra:

memu unnamu naayanamma


vuncle mee email id ivvandi... confluent vaallu helping chesthunnaraa? leka full internal aaa?
 

Netra
Megastar
Username: Netra

Post Number: 25731
Registered: 01-2008
Posted From: 69.53.240.22

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:25 pm:       


Gandhiguevara:

evaranna vaaduthunnaaraa?




memu unnamu naayanamma
YSR AMAR RAHE
 

Emc2
Legend
Username: Emc2

Post Number: 50874
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:23 pm:       


Gandhiguevara:




40 waste ayithe 39 iyyi chalu, kani oka sample resume, interview lekunda 39 ki oka job in kafka ipinchu chalu

remaining nuvu unchuko
 

Heineken
Side Hero
Username: Heineken

Post Number: 3862
Registered: 04-2009
Posted From: 8.39.118.254

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:19 pm:       


Gandhiguevara:


Salesforce ppl
 

Gandhiguevara
Legend
Username: Gandhiguevara

Post Number: 61744
Registered: 10-2009
Posted From: 67.191.221.7

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:18 pm:       


Emc2:

sample resume on kafka unte pampinchu


ilaanti kochens ke neeku 40 kudaa waste anedhi...
 

Emc2
Legend
Username: Emc2

Post Number: 50873
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:16 pm:       


Gandhiguevara:



i like your openness,

sample resume on kafka unte pampinchu
 

Siloan
Legend
Username: Siloan

Post Number: 48025
Registered: 03-2008
Posted From: 132.174.20.41

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:16 pm:       

nayyaldi
 

Gandhiguevara
Legend
Username: Gandhiguevara

Post Number: 61743
Registered: 10-2009
Posted From: 67.191.221.7

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:13 pm:       


Emc2:

billing entha ani sannayi asking


okka kafka ke billing icheyyaru...naaku just $154/Hr C2C isthunnaaru....neeku antha ivvakapovachu
 

Emc2
Legend
Username: Emc2

Post Number: 50871
Registered: 03-2008
Posted From: 173.66.111.159

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:12 pm:       

billing entha ani sannayi asking

depends on the rate vaduthunnaro ledo chepthadu anta
 

Gandhiguevara
Legend
Username: Gandhiguevara

Post Number: 61742
Registered: 10-2009
Posted From: 67.191.221.7

Rating: N/A
Votes: 0 (Vote!)

Posted on Wednesday, September 28, 2016 - 01:11 pm:       

evaranna vaaduthunnaaraa?

Add Your Message Here
Post:
Bold text Italics Underline Create a hyperlink Insert a clipart image HASH(0x987336c){Movie Clipart}
Show / hide regular icons selection options

Click on following links to open cliparts by Alphabetical Order

 A   B   C   D   E   F   G   H   I   J   K   L   M  

N   O   P   Q   R   S   T   U   V   W   X   Y   Z  

Show / Hide Filmy icons selection options

Click on following links to open cliparts by Alphabetical Order

  A   B   C   D   E   F   G   H   I   J   K   L   M  

N   O   P   Q   R   S   T   U   V   W   X   Y   Z  

Username: Posting Information:
This is a public posting area. Enter your username and password if you have an account. Otherwise, enter your full name as your username and leave the password blank. Your e-mail address is optional.
Password:
E-mail:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: