The (local!) TextFormattingRules are applied, but some of the link types (such as WikiNames?) are disabled, since on the remote site, these pointed to remote pages, and now they would point to local pages.
If the remote site produces HTML, then the HTML will appear quoted -- and seem very unreadable. So don't do that, then.
<include "https://www.usemod.org/cgi-bin/mb.pl?action=browse&id=WikiPatches&raw=1">
First, we need to replace the tag in CommonMarkup. Add it after all the quoting rules, ie. after nowiki, pre, and code. In the following example, it is added before the RSS stuff from WikiPatches/RssInclusion, such that the raw stuff being included may contain an RSS include which will be honored.
# The nowiki tag stores text with no markup (except quoting HTML) s/\<nowiki\>((.|\n)*?)\<\/nowiki\>/&StoreRaw($1)/ige; # The pre tag wraps the stored text with the HTML pre tag s/\<pre\>((.|\n)*?)\<\/pre\>/&StorePre($1, "pre")/ige; s/\<code\>((.|\n)*?)\<\/code\>/&StorePre($1, "code")/ige; # The <include "uri..."> includes the raw output of the remote site s/\<include +"(.*)"\>/&IncludeRaw($1)/ige; # The <rss "uri..."> stores the parsed RSS of the given URI s/\<rss +"(.*)"\>/&StoreRss($1)/ige;
Now all we need is this IncludeRaw sub. It uses LWP::UserAgent. Note that we need to apply any text formatting rules to the raw text that have already been applied to the rest of the text at the point in CommonMarkup where we called IncludeRaw.
sub IncludeRaw { require LWP::UserAgent; my ($uri) = @_; my $ua = LWP::UserAgent->new; # consider setting $ua->max_size(50000); # consider setting $ua->timeout(20); my $request = HTTP::Request->new('GET', $uri); my $response = $ua->request($request); my $data = $response->content; # Now repeat some of the text formatting rules here that we already # applied to the raw text. $data = &QuoteHtml($data); $data =~ s/\\ *\r?\n/ /g; # Join lines with backslash at end $data =~ s/\<nowiki\>((.|\n)*?)\<\/nowiki\>/&StoreRaw($1)/ige; $data =~ s/\<pre\>((.|\n)*?)\<\/pre\>/&StorePre($1, "pre")/ige; $data =~ s/\<code\>((.|\n)*?)\<\/code\>/&StorePre($1, "code")/ige; # Now prevent some of the stuff from the remote site to create local links if ($FreeLinks) { $data =~ s/\[\[$FreeLinkPattern\|([^\]]+)\]\]/&StoreRaw($2)/geo; $data =~ s/\[\[$FreeLinkPattern\]\]/&StoreRaw($1)/geo; } if ($BracketText) { $data =~ s/\[$InterLinkPattern\s+([^\]]+?)\]/&StoreRaw($2)/geos; if ($WikiLinks && $BracketWiki) { $data =~ s/\[$LinkPattern\s+([^\]]+?)\]/&StoreRaw($2)/geos; } } $data =~ s/\[$InterLinkPattern\]/&StoreRaw($1)/geo; if ($WikiLinks) { $data =~ s/$LinkPattern/&StoreRaw($1)/geo; } return $data; }
# The <include "uri..."> includes the raw output of the remote site s/\<include +"(.*)"\>/&IncludeRaw($1)/ige;
This makes the inclusion recursive, but now I do need to be careful about loops. I should really add node-checking to make sure a loop doesn't exist. -- ElizabethDalton?
After writing the above, I discovered that I was using wiki.pl to resolve the includes, not cwick.pl. The inclusion patch works in cwick, but the raw mode doesn't.