twitter4j - Twitter data streaming - dynamic keyword value change -


i have doing sentiment analysis on twitter data in hadoop.

i have configured flume bring data based upon keywords, have maintained in "flume.conf" file, shown below 

twitteragent.sources.twitter.keywords = kitkat, nescafe, carnation, milo, cerelac

with mentioning this, know whether possible dynamically change keywords, based upon java program pop-up asking keywords user.

also, apart method, please provide other methods folks suggest hide complexity of updating flume.conf file.

best regards, ram

flume provides application class run configuration file using java program. 

public class flume{ public static void main(string[] args) throws exception { new conf().setconfiguration();   string[] args1 = new string[] { "agent","-ntwitteragent",   "-fflume.conf" }; basicconfigurator.configure(); application.main(args1); system.setproperty("hadoop.home.dir", "/");  } }

here change edit conf file using file utils this

class conf{ int ch; string keyword =""; scanner sc= new scanner(system.in); public void setconfiguration(){     system.out.println("enter keyword");     keyword=sc.nextline();     byte[] key= keyword.getbytes();      fileoutputstream fp=null;     fileinputstream src=null;      try{         fp= new fileoutputstream("flumee.conf");         src= new fileinputstream("flume.conf");         while((ch=src.read())!=-1){             fp.write((char)ch);         }          fp.write(key);      }catch(exception e){         system.out.println("file exception:"+ e);     }finally{         try{             if(fp!=null){                 fp.close();             }             if(src!=null){                 src.close();             }         }catch(exception e){             system.out.println("file closing exception:"+ e);         }     } } }

now need keep twitteragent.sources.twitter.keywords= line @ end of flume configuration file easier add keywords file. flume.conf file looks this

 twitteragent.sources= twitter  twitteragent.channels= memchannel  twitteragent.sinks=hdfs  twitteragent.sources.twitter.type = com.cloudera.flume.source.twittersource  twitteragent.sources.twitter.channels=memchannel   twitteragent.sources.twitter.consumerkey=xxx  twitteragent.sources.twitter.consumersecret=xxx  twitteragent.sources.twitter.accesstoken=xxx  twitteragent.sources.twitter.accesstokensecret=   twitteragent.sinks.hdfs.channel=memchannel  twitteragent.sinks.hdfs.type=hdfs  twitteragent.sinks.hdfs.hdfs.path=hdfs://localhost:9000/user/flume/direct  twitteragent.sinks.hdfs.hdfs.filetype=datastream  twitteragent.sinks.hdfs.hdfs.writeformat=text  twitteragent.sinks.hdfs.hdfs.batchsize=1000  twitteragent.sinks.hdfs.hdfs.rollsize=0  twitteragent.sinks.hdfs.hdfs.rollcount=10000  twitteragent.sinks.hdfs.hdfs.rollinterval=600  twitteragent.channels.memchannel.type=memory  twitteragent.channels.memchannel.capacity=10000  twitteragent.channels.memchannel.transactioncapacity=1000   twitteragent.sources.twitter.keywords=

when run program ask keywords first , start collecting tweets keyword.


Comments

Post a Comment

Popular posts from this blog

Android layout hidden on keyboard show -

i have strange problem layout. when tap on edittext , keyboard shown edittext stay on bottom , when tap on to hide keyboard , keyboard hides , edittext go should when want input text. when tap again, keyboard shown , edittext goes down behind keyboard... <relativelayout android:layout_width="match_parent" android:layout_height="match_parent" > <scrollview android:layout_width="match_parent" android:layout_height="fill_parent" android:above="@+id/rl_commands"> <textview android:layout_width="match_parent" android:layout_height="wrap_content"/> </scrollview> <relativelayout android:layout_width="match_parent android:layout_height="24dp" android:id="@+id/rl_commands" android:alignparentbottom="true">
Read more

google app engine - 403 Forbidden POST - Flask WTForms -

i have flask application running on google app engine. trying submit form built using wtform , keep getting following error. 403 forbidden: don't have permission access requested resource. either read-protected or not readable server. location.html (part of code) <form method=post action="/home/location"> .... <button type="submit" class="btn btn-primary" value="submit"> directions </button> </form> main.py @app.route('/home/location', methods=['post', 'get']) def location(): form = cfcdirections.direction(request.form) print(request.method) if request.method == 'post' , form.validate(): print("after if") directions = cfcdirections() street_no = directions.no street = directions.street suburb = directions.suburb postcode = directions.postcode
Read more

c - Why would PK11_GenerateRandom() return an error -8023? -

i looking through internet trying find source of pk11_generaterandom() function see why function fail. have program uses function when moved new flavor of linux, fails after forking ( fork() ) since not believe there problem nss, suspect doing incorrectly disregarded in older versions of linux new 1 there issue. the openssl package same on 'good' , 'bad' server: openssl 0.9.8e-fips-rhel5 01 jul nss rpm differs though. 'good' has nss-3.12.2.0-2.el5 and bas has version nss-3.15.3-4.el5_10 the 'good' server uses quite obsolete linux: linux 2.6.18-128.el5 #1 smp wed jan 21 08:45:05 est 2009 x86_64 x86_64 x86_64 gnu/linux enterprise linux enterprise linux server release 5.3 (carthage) red hat enterprise linux server release 5.3 (tikanga) the 'bad' server newer: linux bad 2.6.18-371.4.1.el5 #1 smp wed jan 29 11:05:49 pst 2014 x86_64 x86_64 x86_64 gnu/linux oracle linux server release 5.10 red hat enterprise linux server release
Read more