By FordLawnmower on Feb 16, 2013Gathers results from rss feed pages and returns them to the channel.
Partyline Commands:
Syntax/Example:
.rss add #Hawkee,#mIRC hawkee \002\00307,14H\00301,14awkee\017 http://www.hawkee.com/comment.rss.php?tool_type=snippet_id
ScreenShot:
Loading:
See the setup at the top of the script for more options.
##############################################################################################
## ## rssreader.tcl for eggdrop by Ford_Lawnmower irc.geekshed.net #Script-Help ## ##
##############################################################################################
## .rss in the party line for commands and syntax. Example Feed Add Below: ##
##.rss add #Hawkee hawkee Hawkee http://www.hawkee.com/comment.rss.php?tool_type=snippet_id ##
##############################################################################################
## ____ __ ########################################### ##
## / __/___ _ ___ _ ___/ /____ ___ ___ ########################################### ##
## / _/ / _ `// _ `// _ // __// _ \ / _ \ ########################################### ##
## /___/ \_, / \_, / \_,_//_/ \___// .__/ ########################################### ##
## /___/ /___/ /_/ ########################################### ##
## ########################################### ##
##############################################################################################
## ## Start Setup. ## ##
##############################################################################################
namespace eval rssreader {
## Edit textf to change the color/state of the text shown ## ##
variable textf "\017\00304"
## Edit linkf to change the color/state of the links ## ##
variable linkf "\017\037"
## Change usetiny to shorten links with tinyurl. 1 = on , 0 = off ## ##
variable usetiny 1
## Change useisgd to shorten links with is.gd. 1 = on , 0 = off ## ##
variable useisgd 0
## Edit maxresults to the amount of results you want per query. This will not cause ## ##
## You to lose results. It will only spread them out over several querys. ## ##
variable maxresults 2
## Edit checkdelay to change the frequency feed pages are checked. Delay is in minutes. ## ##
variable checkdelay 3
## Edit startupdelay to add delay to startup/restart. Delay is in minutes. ## ##
variable startupdelay 2
##############################################################################################
## ## End Setup. ## ##
##############################################################################################
bind dcc - rss rssreader::settings
bind evnt -|- init-server rssreader::loadhash
bind evnt -|- prerehash rssreader::loadhash
proc settings {hand idx text} {
set choice [lindex $text 0]; set channel [lindex $text 1]
set name [lindex $text 2]; set logo [lindex $text 3]
set link [lindex $text 4]
if {[string equal -nocase "add" $choice] && $link != ""} {
set deleteinfo [string map {\[ ? \] ?} [hget "rssreader" $name]]
if {[set indx [lsearch -glob [timers] "*$deleteinfo*"]] != -1 && $deleteinfo != "0"} {
killtimer [lindex [lindex [timers] $indx] 2]
}
hadd "rssreader" "$name" "rssreader::main {$channel $logo $link}"
savehash
putdcc $idx "Added feed $name to $channel as $logo $link"
rssreader::main "$channel $logo $link"
} elseif {[string equal -nocase "addxml" $choice] && $link != ""} {
set deleteinfo [string map {\[ ? \] ?} [hget "rssreader" $name]]
if {[set indx [lsearch -glob [timers] "*$deleteinfo*"]] != -1 && $deleteinfo != "0"} {
killtimer [lindex [lindex [timers] $indx] 2]
}
hadd "rssreader" "$name" "rssreader::type2 {$channel $logo $link}"
savehash
putdcc $idx "Added feed $name to $channel as $logo $link"
rssreader::type2 "$channel $logo $link"
} elseif {[string equal -nocase "addssl" $choice] && $link != ""} {
set deleteinfo [string map {\[ ? \] ?} [hget "rssreader" $name]]
if {[set indx [lsearch -glob [timers] "*$deleteinfo*"]] != -1 && $deleteinfo != "0"} {
killtimer [lindex [lindex [timers] $indx] 2]
}
hadd "rssreader" "$name" "rssreader::type3 {$channel $logo $link}"
savehash
putdcc $idx "Added feed $name to $channel as $logo $link"
rssreader::type3 "$channel $logo $link"
} elseif {[string equal -nocase "list" $choice]} {
putdcc $idx "\[RSS list\]"
set count [hfind "rssreader" "*" 0]; set counter 1
while {$count >= $counter} {
putdcc $idx "[hfind "rssreader" "*" $counter]"
incr counter
}
} elseif {[string equal -nocase "info" $choice] && $channel != ""} {
putdcc $idx "[hget "rssreader" $channel]"
} elseif {[string equal -nocase "delete" $choice] && $channel != ""} {
set deleteinfo [string map {\[ ? \] ?} [hget "rssreader" $channel]]
hdel "rssreader" $channel
savehash
if {[set indx [lsearch -glob [timers] "*$deleteinfo*"]] != -1 && $deleteinfo != "0"} {
killtimer [lindex [lindex [timers] $indx] 2]
putdcc $idx "Removed $channel from RSS"
} else {
putdcc $idx "$channel not found"
}
} elseif {[string equal -nocase "timers" $choice]} {
putdcc $idx [timers]
} elseif {[string equal -nocase "rehash" $choice]} {
putdcc $idx "rehashing rss...."
hfree rsstempold
hfree rsstempnew
putdcc $idx "done"
} else {
putdcc $idx "\[RSS Syntax\]"
putdcc $idx "Add Feed: .rss add <#chan1,#chan2,#chanetc> <name> <logo> <link>"
putdcc $idx "Add Xml Feed: .rss addxml <#chan1,#chan2,#chanetc> <name> <logo> <link>"
putdcc $idx "Add SSL Feed: .rss addssl <#chan1,#chan2,#chanetc> <name> <logo> <link>"
putdcc $idx "Delete Feed: .rss delete <name>"
putdcc $idx "Info Feed: .rss info <name>"
putdcc $idx "List Feeds: .rss list"
putdcc $idx "Help Feeds: .rss"
}
}
proc main {text} {
set chan [lindex $text 0]; set logo [lindex $text 1]; set linker [lindex $text 2]
set title ""; set link ""; set description ""; set maxcount 1; set json ""
if {[set indx [lsearch -glob [timers] "*rssreader::main {$chan [string map {\[ ? \] ?} $logo] ${linker}}*"]] != -1} {
killtimer [lindex [lindex [timers] $indx] 2]
}
timer $rssreader::checkdelay "rssreader::main {$chan $logo $linker}"
set rssreaderurl "/ajax/services/feed/load?v=1.0&q=${linker}"
set rssreadersite "ajax.googleapis.com"; set rssout ""
if {[catch {set rssreadersock [socket -async $rssreadersite 80]} sockerr]} {
return 0
} else {
puts $rssreadersock "GET $rssreaderurl HTTP/1.0"
puts $rssreadersock "Host: $rssreadersite"
puts $rssreadersock "User-Agent: Opera 9.6"
puts $rssreadersock "Connection: close"
puts $rssreadersock ""
flush $rssreadersock
while {![eof $rssreadersock]} {
set rssreadervar " [string map {<![CDATA[ "" ]]> "" \$ \002\$\002 \[ \( \] \)} [gets $rssreadersock]] "
if {[regexp {\"responseStatus\":\s?400} $rssreadervar]} {
if {[set indx [lsearch -glob [timers] "*rssreader::main {$chan $logo ${linker}}*"]] != -1} {
killtimer [lindex [lindex [timers] $indx] 2]
}
type2 "$chan $logo $linker"
close $rssreadersock
return
} else {
regexp {\:\[(\{.*)$} $rssreadervar match rssout
set rssout [regexp -all -inline {\{(.*?)\}} $rssout]
if {$rssout != ""} {
set count 0
foreach {match matched} $rssout {
incr count
set matched [regexp -all -inline {(".*?":".*?"\,)} $match]
foreach {innermatch innermatched} $matched {
regexp {\"(.*?)\":\".*?\"\,} $innermatch match varname
regexp {\".*?\":\"(.*?)\"\,} $innermatch match value
set value [string map {\$ \002\$\002 \] \002\]\002 \[ \002\[\002} $value]
set $varname $value
}
if {[hfindexact "rsstempold" "${link}" 1] != $link} {
if {$title == ""} { set title $description }
set linked $link
if {$rssreader::usetiny} { set linked [string trimright [tiny $link]] }
if {$rssreader::useisgd} { set linked [string trimright [isgd $link]] }
if {$maxcount <= $rssreader::maxresults} {
putserv "PRIVMSG $chan :${logo} ${rssreader::textf}[dehex $title] ${rssreader::linkf}${linked}"
incr maxcount
hadd "rsstempnew" $link 1
}
}
}
hfree rsstempold
hcopy rsstempnew rsstempold
rssreader::savetemphash
}
}
}
close $rssreadersock
}
}
proc type2 {text} {
set chan [lindex $text 0]; set logo [lindex $text 1]; set linker [lindex $text 2]
set title ""; set link ""; set description ""
if {[set indx [lsearch -glob [timers] "*rssreader::type2 {$chan [string map {\[ ? \] ?} $logo] ${linker}}*"]] != -1} {
killtimer [lindex [lindex [timers] $indx] 2]
}
timer $rssreader::checkdelay "rssreader::type2 {$chan $logo $linker}"
regexp -- {https?\:\/\/(.*?)(\/.*)$} $linker wholematch rsstype2site rsstype2url
set itemfound 0 ; set maxcount 1
if {[catch {set rsstype2sock [socket -async $rsstype2site 80]} sockerr]} {
return 0
} else {
puts $rsstype2sock "GET $rsstype2url HTTP/1.0"
puts $rsstype2sock "Host: $rsstype2site"
puts $rsstype2sock "User-Agent: Opera 9.6"
puts $rsstype2sock "Connection: close"
puts $rsstype2sock ""
flush $rsstype2sock
while {![eof $rsstype2sock]} {
set rsstype2var " [string map {<![CDATA[ "" ]]> "" \$ \002\$\002 \[ \( \] \)} [gets $rsstype2sock]] "
if {[string match {*<item>*} $rsstype2var]} { set itemfound 1 }
if {[regexp {<title>(.*?)(?:<\/title>|$)} $rsstype2var match title]} { }
if {[regexp {<link>(.*?)(?:<\/link>|$)} $rsstype2var match link]} { }
if {[string match {*</item>*} $rsstype2var]} {
if {[hfindexact "rsstempold" "${link}" 1] != $link} {
if {$itemfound} {
if {$maxcount <= $rssreader::maxresults} {
set linked $link
if {$rssreader::usetiny} { set linked [string trimright [tiny $link]] }
if {$rssreader::useisgd} { set linked [string trimright [isgd $link]] }
putserv "PRIVMSG $chan :${logo} ${rssreader::textf}[dehex $title] ${rssreader::linkf}${linked}"
incr maxcount
hadd "rsstempnew" $link 1
}
}
}
}
}
set itemfound 0
hfree rsstempold
hcopy rsstempnew rsstempold
rssreader::savetemphash
close $rsstype2sock
}
}
proc type3 {text} {
set chan [lindex $text 0]; set logo [lindex $text 1]; set linker [lindex $text 2]
set title ""; set link ""; set description ""
if {[set indx [lsearch -glob [timers] "*rssreader::type3 {$chan [string map {\[ ? \] ?} $logo] ${linker}}*"]] != -1} {
killtimer [lindex [lindex [timers] $indx] 2]
}
timer $rssreader::checkdelay "rssreader::type3 {$chan $logo $linker}"
regexp -- {https?\:\/\/(.*?)(\/.*)$} $linker wholematch rsstype3site rsstype3url
set itemfound 0 ; set maxcount 1 ; set count 1 ; set counter 1
::http::register https 443 tls::socket
set type3searchtoken [::http::geturl "https://${rsstype3site}${rsstype3url}"]
set type3searchdata [string map {<!\[CDATA\[ \"\" \]\]> "" \$ \\\$ \[ \\\[ \] \\\]} [::http::data $type3searchtoken]]
::http::cleanup $type3searchtoken
::http::unregister https
set title [regexp -all -inline {<title>(.*?)<\/title>} $type3searchdata]
set link [regexp -all -inline {<link>(.*?)<\/link>} $type3searchdata]
while {$counter <= [llength $title]} {
set thislink [lindex ${link} $counter]
if {[hfindexact "rsstempold" "${thislink}" 1] != $thislink} {
if {$maxcount <= $rssreader::maxresults} {
set linked $thislink
if {$rssreader::usetiny} { set linked [string trimright [tiny $thislink]] }
if {$rssreader::useisgd} { set linked [string trimright [isgd $thislink]] }
putserv "PRIVMSG $chan :${logo} ${rssreader::textf}[dehex [lindex $title $counter]] ${rssreader::linkf}${linked}"
hadd "rsstempnew" $thislink 1
incr maxcount 1
}
incr count 2
}
incr counter 2
}
hfree rsstempold
hcopy rsstempnew rsstempold
rssreader::savetemphash
}
proc tiny {link} {
set tinysite tinyurl.com
set tinyurl /api-create.php?url=[urlencode ${link}]
if {[catch {set tinysock [socket -async $tinysite 80]} sockerr]} {
putlog "$tinysite $tinyurl $sockerr error"
return $link
} else {
puts $tinysock "GET $tinyurl HTTP/1.0"
puts $tinysock "Host: $tinysite"
puts $tinysock "User-Agent: Opera 9.6"
puts $tinysock "Connection: close"
puts $tinysock ""
flush $tinysock
while {![eof $tinysock]} {
set tinyvar " [gets $tinysock] "
if {[regexp {(http:\/\/.*)} $tinyvar match tinyresult]} {
close $tinysock
return [string map {http:// https://} $tinyresult]
}
}
close $tinysock
return $link
}
}
proc isgd {link} {
set isgdsite is.gd
set isgdurl /create.php?format=simple&url=[urlencode ${link}]
if {[catch {set isgdsock [socket -async $isgdsite 80]} sockerr]} {
putlog "$isgdsite $isgdurl $sockerr error"
return $link
} else {
puts $isgdsock "GET $isgdurl HTTP/1.0"
puts $isgdsock "Host: $isgdsite"
puts $isgdsock "User-Agent: Opera 9.6"
puts $isgdsock "Connection: close"
puts $isgdsock ""
flush $isgdsock
while {![eof $isgdsock]} {
set isgdvar " [gets $isgdsock] "
if {[regexp {(http:\/\/.*)} $isgdvar match isgdresult]} {
close $isgdsock
return $isgdresult
}
}
close $isgdsock
return $link
}
}
proc hex {decimal} { return [format %x $decimal] }
proc decimal {hex} { return [expr 0x$hex] }
proc dehex {string} {
regsub -all {^\{|\}$} $string "" string
set string [subst [regsub -nocase -all {\\u([a-f0-9]{4})} $string {[format %c [decimal \1]]}]]
set string [subst [regsub -nocase -all {\%([a-f0-9]{2})} $string {[format %c [decimal \1]]}]]
set string [subst [regsub -nocase -all {\&#([0-9]{4});} $string {[format %c \1]}]]
set string [subst [regsub -nocase -all {\&#x([0-9]{2});} $string {[format %c [decimal \1]]}]]
set string [string map {" \" · · & & <b> \002 </b> \002 – – » \
» « « Ü Ü ü ü Á Á á á É É é é Í Í í \
í Ó Ó ó ó Ñ Ñ ñ ñ Ú Ú ú ú æ æ " " ' \' \
\( \002\(\002 \) \002\)\002 \{ \002\{\002 \} \002\}\002} $string]
return $string
}
proc urlencode {string} {
regsub -all {^\{|\}$} $string "" string
return [subst [regsub -nocase -all {([^a-z0-9\+])} $string {%[format %x [scan "\\&" %c]]}]]
}
proc hadd {hashname hashitem hashdata } {
global $hashname
set ${hashname}($hashitem) $hashdata
}
proc hget {hashname hashitem} {
upvar #0 $hashname hgethashname
if {[info exists hgethashname($hashitem)]} {
return $hgethashname($hashitem)
} else {
return 0
}
}
proc hfind {hashname search value} {
upvar #0 $hashname hfindhashname
if {[array exists hfindhashname]} {
if {$value == 0} {
return [llength [array names hfindhashname $search]]
} else {
set value [expr $value - 1]
return [lindex [array names hfindhashname $search] $value]
}
}
}
proc hfindexact {hashname search value} {
upvar #0 $hashname hfindhashname
if {[array exists hfindhashname]} {
if {$value == 0} {
return [llength [array names hfindhashname -exact $search]]
} else {
set value [expr $value - 1]
return [lindex [array names hfindhashname -exact $search] $value]
}
}
}
proc hsave {hashname filename} {
upvar #0 $hashname hsavehashname
if {[array exists hsavehashname]} {
set hsavefile [open $filename w]
foreach {key value} [array get hsavehashname] {
puts $hsavefile "${key}=${value}"
}
close $hsavefile
}
}
proc hload {hashname filename} {
upvar #0 $hashname hloadhashname
hfree $hashname
set hloadfile [open $filename]
set linenum 0
while {[gets $hloadfile line] >= 0} {
if {[regexp -- {([^\s]+)=(.*)$} $line wholematch item data]} {
set hloadhashname($item) $data
}
}
close $hloadfile
}
proc hfree {hashname} {
upvar #0 $hashname hfreehashname
if {[array exists hfreehashname]} {
foreach key [array names hfreehashname] {
unset hfreehashname($key)
}
}
}
proc hdel {hashname hashitem} {
upvar #0 $hashname hdelhashname
if {[info exists hdelhashname($hashitem)]} {
unset hdelhashname($hashitem)
}
}
proc hcopy {hashfrom hashto} {
upvar #0 $hashfrom hashfromlocal $hashto hashtolocal
array set hashtolocal [array get hashfromlocal]
}
proc savetemphash {} {
#hsave "rsstempnew" "${::network}rsstemp.hsh"
}
proc savehash {} {
hsave "rssreader" "${::network}rssreader.hsh"
}
proc loadhash {type} {
if {[file exists "${::network}rssreader.hsh"]} {
rssreader::hload "rssreader" "${::network}rssreader.hsh"
}
if {[file exists "${::network}rsstemp.hsh"]} {
#rssreader::hload "rsstempnew" "${::network}rsstemp.hsh"
#rssreader::hload "rsstempold" "${::network}rsstemp.hsh"
}
set count $rssreader::startupdelay
foreach {key value} [array get ::rssreader] {
if {[set indx [lsearch -glob [timers] "*$value*"]] != -1} {
killtimer [lindex [lindex [timers] $indx] 2]
}
timer $count $value
incr count
}
}
}
putlog "\002*Loaded* \017\00304\002RSS Reader\017 \002by \
Ford_Lawnmower irc.GeekShed.net #Script-Help .rss for help"
Works great in 2017 too :)
Just need some changes like, adding
package require tls
package require http
if needed, and change on line 223 where
http::register https 443 tls::socket
to
if { [info exists {my-ip}] == 1 && [string length ${my-ip}] > 0} {
http::register https 443 [list tls::socket -myaddr ${my-ip} -tls1 1]
} elseif { [info exists {my-hostname}] == 1 && [string length ${my-hostname}] > 0} {
http::register https 443 [list tls::socket -myaddr ${my-hostname} -tls1 1]
} else {
http::register https 443 [list tls::socket -tls1 1]
}
Hi Ford:
I have been using this script and noticed that when adding a feed via addxml, it works flawlessly. However, when trying to add another via feed via plain add (which, as per my my understanding, loads the feed via google apis) the script is added, but no updates at all are sent to the channel. Is it possible that the parsing for the main function is broken?
Hi
Thanks for your work man :)
I tried it and it's work only for .rss addxml
But my Q is:
Can I make a delay between rss message?
When I start the bot it's bring all the news from all rss feed I add.. it's spam really.. I could delay when it's start to bring the news but when it start it won't stop until he bring it all
This script work great for an high up time computer or for a server.. but with my regular computer how disconnect once every day it's spam really :(
If you could make an option to make a delay between rss feed message.. or make the script bring only last 3 rss feed and start bring any new one after that.. it would be greaat
Anyway I must thank you very much for all your work.. I said it and say it again.. you are the master :)
Regards,
Libbard
Hello there,
I've posted the source, and the link to the current page, on a french eggdrop community: http://forum.eggdrop.fr/Lecteur-RSS-t-1328.html
If our users made some modifications, I'll post them here.
Regards !
Thanks for this script, but it doesn't works with my link, can you check this? the link is http://legendas.tv/rss.html, I've tried to use .rss add... and .rss addxml, the script retrieve just two lines
(17:43:31) [Lstv] Touch.S02E09.HDTV.XviD-AFG-LOL-DIMENSION-mSD-ECI http://tinyurl.com/3cgrd2
(17:43:32) [Lstv] Beauty.and.the.Beast..S01E17.HDTV.XviD-AFG/EVOLVE/IMMERSE/WEB-DL.mSD/KiNGS http://tinyurl.com/3cgrd2
ando don't get more news from the feed. thanks in advance.
possible to make it auto announce the RSS once something has been updated ?
i get this <!(CDATA((GPGT) Joanne 'piaked' and thrown away by tiko Ang Moh player. LMAO))> http://tinyurl.com/bvmwjmu
for every RSS feed. the <!(CDATA is infront of every feed. could i remove that ?
For some strange reason this feed stops/starts/stops/starts posting to the channel. I have it type 2:http://feeds.marketwatch.com/marketwatch/realtimeheadlines. When you get a chance could you look and see what the problem might be.
Can't seem to get this feed to work...
rssreader::main {#options [Market-Pulse] http://www.marketwatch.com/rss/marketpulse}
I get the following with the new code:
dutch> .rss add #options mktpulse [\0034Market-Pulse\003] http://feeds.marketwatch.com/marketwatch/marketpulse/
[10:45:56] Added feed mktpulse to #options as [Market-Pulse] http://feeds.marketwatch.com/marketwatch/marketpulse/ [10:45:56] Tcl error [rssreader::settings]: can't read "rssreadervar": no such variable
Updated script and did not get any errors however not getting any posts.. This is the only change I did to your script:
variable textf "\017\02"
variable linkf "\017\0312"
this is what is inside the dalnetrssreader.hsh file:
long=rssreader::main {#stocktrader [4Long-Ideas] http://seekingalpha.com/tag/long-ideas.xml}
mktpulse=rssreader::main {#stocktrader [4Market-Pulse] http://feeds.marketwatch.com/marketwatch/marketpulse/}
Just updated this script. If you are using the original version.
Sorry for the large update. It was needed to adapt the script for different types of feeds.
@rinzes If you are having trouble with a feed, please post the feed link here so I can look at it. RSS pages are highly subjective because lots of feeds like to take the Simple out and replace it with Complicated.
Edit ** Added a command to use a second parser. If you have trouble with a feed try .rss addxml instead of .rss add
@FordLawnmower Try this: http://codemirror.net/demo/loadmode.html You can paste your code here and play with the different modes. I'm currently using perl for TCL, but it's not quite right as we can see. If you want to create a new mode I'd download the entire library, install it locally, copy the best match and modify it.
Little confused on how this should work. It's supposed to check the RSS feeds every so often (specified in the conf file) and then post them to the channel, correct?
I tried .rss addxml #AWPWX hunchat HUNCHAT http://mesonet.agron.iastate.edu/iembot-rss/wfo/khun.xml, it said the feed was added but it hasn't been posting in #AWPWX. The code remains unchanged except for the variables I am supposed to edit. I have it check the feeds every minute. http://pastie.org/9096753
Yea mine does the same :S Got any fix?
My feed is the following http://www.reddit.com/r/TheRealmOfMianite/new.rss yet all I can get it to post is on startup it shows
" Reddit newest submissions : TheRealmOfMianite http://www.reddit.com/r/TheRealmOfMianite/"
What am I doing wrong?