WikiProject iconPortals  
WikiProject iconThis page is within the scope of WikiProject Portals, a collaborative effort to improve portals on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.PortalsWikipedia:WikiProject PortalsTemplate:WikiProject PortalsPortals articles
Template This module does not require a rating on the project's quality scale.
Note icon
See also: List of Portals

Should featured articles also use excerpts?

Could featured articles also use excerpts? I think yes but other editors have told me "no" because if you have the featured article on your watchlist you would not be notified if someone changed something in the transcluded article section that is embedded in the featured article. Could we make it more palatable if a change in the transcluded section results in a notification in someone's watchlist? So if e.g. climate change had transcluded the lead of climate change in Africa, and something was changed in climate change in Africa then it would pop up in my watchlist even if I only have climate change in my watchlist but not climate change in Africa? EMsmile (talk) 12:49, 23 August 2021 (UTC)[reply]

Without spilling the beans explicitly, excerpts in protected articles may also leave scope for abuse. Alternatively, they may result in excessive cascading protection. I think the benefits outweigh the problems, but wonder how closely this aspect has been considered. Certes (talk) 13:07, 23 August 2021 (UTC)[reply]
We don't use "cascading protection" on articles, and this shouldn't lead to that. Pseduo-cascade protection (where a string of transcluded pages all get manually protected) could be an issue though. — xaosflux Talk 18:54, 23 August 2021 (UTC)[reply]
Not a serious problem then. I knew I'd seen cascade-protected pages (probably templates), but hadn't realised that the protection was due to transclusion on today's main page. Certes (talk) 19:12, 23 August 2021 (UTC)[reply]
I just found there's a cascade-watching feature request on Phabricator and added excerpts as another use case. Sophivorus (talk) 18:47, 23 August 2021 (UTC)[reply]
Great. What are the chances that it will get programmed? I noticed it was first requested in 2013 and has been lingering for the last 8 years? Is there anything else we can do to lend it more support? EMsmile (talk) 00:07, 24 August 2021 (UTC)[reply]
After my comment, the task was tagged as in need of review to determine prioritization by the Growth team. I think the best we can do is add reasons why it should be prioritized. Sophivorus (talk) 01:06, 24 August 2021 (UTC)[reply]
I have added a comment on Phabricator, hope it helps. I wasn't sure if that page is similar to a talk page or how it works. Maybe it would be better to start up a new task proposal rather than trying to rekindle the one from 2013? That old task doesn't even mention excerpt in the title. It's called "Cascade-watching for transcluded pages". EMsmile (talk) 02:07, 24 August 2021 (UTC)[reply]
Great comment, it'll definitely help! As to starting a new task, per my knowledge of the software, I think that task is very much spot-on, and the fact that the original request was motivated by another use case may actually be beneficial. In fact, I just left another comment on Phabricator describing yet another use case for "cascade-watching" and tagging the Editing Team. Sophivorus (talk) 11:45, 24 August 2021 (UTC)[reply]

@EMsmile, Sdkb, Certes, Xaosflux Hi guys! I just proposed this feature for the 2022 Wishlist. I thought you may want to know, cheers! Sophivorus (talk) 13:37, 19 January 2022 (UTC)[reply]

thanks, Sophivorus, I've placed a "support" vote on that page. Thanks for following up on this. EMsmile (talk) 10:35, 4 February 2022 (UTC)[reply]

Short description being transcluded

The template's documentation says that templates listed at Module:Excerpt/config will not be transcluded, and ((Short description)) appears to meet those criteria, but in this version of my sandbox, an excerpt of House clearly transcludes the short description. You can also see the short description if you feed an excerpt to Special:ExpandTemplates. I did some experimenting, and it appears that using the |templates= option with a hyphen transcludes all templates except those specifically excluded, which appears to conflict with the clause I quoted above. I have updated the documentation to match the template's current behavior. How do I specify ((Excerpt)) so that it excludes all of the templates in /config and also excludes specific templates of my choice? – Jonesey95 (talk) 21:44, 14 December 2021 (UTC)[reply]

@Jonesey95 Hi! Currently there's no way to do what you want, namely to add a template to the blacklist (unless of course you copy the entre blacklist to the parameter). However, it's a frequent request (see for example the discussion right above us) and I'd very much like to implement a solution, I just haven't come up with a good syntax yet. I'm thinking maybe introducing the + sign in some way, like for example templates=+Foo would add Template:Foo to the blacklist, but then again, using a + sign to exclude a template isn't exactly intuitive. Perhaps the ~ sign instead? Any ideas? Sophivorus (talk) 00:08, 15 December 2021 (UTC)[reply]
We could address both this request and the Coord discussion above with "+" to mean transclude and "-" to mean don't transclude. We also need a token to represent all templates not in Module:Excerpt/config: I'll use "T" here but there must be better choices. It will also be useful to have a token for all templates, even those in /config: I'll use "*". Starting with "-" implicitly transcludes all templates other than those we go on to exclude, i.e. templates=-X is short for templates=* - X. Comma means either "+" or "-": whichever appeared most recently, or "+" if neither appeared. We then have:
  • templates=T — Transclude all templates not in /config: the default value, shown only for illustration
  • templates=T + Coord — Transclude all templates not in /config; also transclude Coord
  • templates=T - Multiple image, About, Living space — Transclude all templates not in /config except Multiple image, About, Living space
  • templates=* — Transclude all templates, even those in /config
The examples in the documentation still work:
  • templates=Infobox — Transclude the template 'Infobox'
  • templates=Infobox, Navbox, Chart — Transclude the template 'Infobox', 'Navbox' and 'Chart'
  • templates=-Infobox — Transclude all templates except 'Infobox' (ignores Module:Excerpt/config)
  • templates=-Infobox, Navbox, Chart — Transclude all templates except 'Infobox', 'Navbox' and 'Chart'
Is that feasible? There will be complications, such as redirects to templates used as aliases. Certes (talk) 01:52, 15 December 2021 (UTC)[reply]
Another option (that I admittedly haven't thought through) would be to use a new operator, like ! (which I chose because it often means "not") to mean "exclude this list of templates and the blacklisted templates", so that templates=!Infobox, Navbox, Chart would work the same as templates=-Infobox, Navbox, Chart except that it would also exclude the blacklist. – Jonesey95 (talk) 06:44, 15 December 2021 (UTC)[reply]
Another gotcha to watch for with my suggestion and others is to avoid punctuation used in template names. (Does Portal-inline mean "Portal but not Inline", or ((Portal-inline))?) There is a theoretical risk to existing uses. Certes (talk) 20:51, 15 December 2021 (UTC)[reply]
Thanks!! Your suggestions are both excellent. I'll let them sink for a few days and I'll write back when I gain some clarity. As a technical note, I think this functionality will be added to Module:Excerpt rather than Module:Transcluder. Let me know of any more ideas!! Sophivorus (talk) 19:50, 16 December 2021 (UTC)[reply]
@Certes @Jonesey95 Hi! After giving it some time and thought, I now think it may actually be enough to simply append the "templates" parameter to the default blacklist if it starts with hyphen (-) and replace it if it doesn't. After all, I can't recall of any actual cases where someone wanted to "exclude templates X and Y but include the ones in the blacklist". Do you? It's always "exclude templates X and Y as well as those in the blacklist". And if in some particular edge case someone wants to do just that, it should actually be quite easy to do by listing all the templates you DO want to transclude, since a normal intro or section usually contains less than 10. So I just finished implementing this functionality at Module:Excerpt/sandbox. Check it out at Template:Excerpt/testcases2#Templates and let me know your thoughts! Sophivorus (talk) 22:37, 20 January 2022 (UTC)[reply]
Looking at the actual usage of |templates=, assuming it's all done via ((Excerpt)):
I think your simplified suggestion deals with all of those cases except the first one, which seems very confusing and could be done better using |templates=coord like False Bay. Certes (talk) 23:14, 20 January 2022 (UTC)[reply]
@Certes As always, thanks for your sharp eye! Looking into List of United States tornadoes in 1950, I realized I could easily solve it by setting templates=.* (meaning transclude all templates). This also gives me some ideas on how to deal with future edge cases. Thus, if no further problems arise, tomorrow I'll deploy the new version. Thanks again, cheers! Sophivorus (talk) 00:42, 21 January 2022 (UTC)[reply]
Deployed! Sophivorus (talk) 21:12, 21 January 2022 (UTC)[reply]

Hatnotes in source article

I think template:excerpt should exclude hatnotes in the source article. See a counter example: Spatial_analysis#Spatial_heterogeneity. fgnievinski (talk) 04:29, 13 January 2022 (UTC)[reply]

@Fgnievinski I just added Template:Multiple issues to the blacklist at Module:Excerpt/config, thanks for the report! Sophivorus (talk) 21:50, 15 January 2022 (UTC)[reply]

Suggestions of templates to blacklist

((Further)), ((Further2)), ((Broader)), ((Outline)), ((Refimprove)), ((SpecialChars)) -- maybe all of Category:Hatnote_templates? I didn't want to edit Module:Excerpt/config. fgnievinski (talk) 18:16, 30 January 2022 (UTC)[reply]

One more: ((Merge from)), ((Merge to)); see example at Ribbon theory. @Sophivorus and Certes: fgnievinski (talk) 19:28, 2 March 2022 (UTC)[reply]

These templates generally appear on a line of their own, so will be excluded from portals. None of them appear there currently, but please ping me if you see one. They may need adding to /config for non-portal excerpts. Certes (talk) 19:47, 2 March 2022 (UTC)[reply]
@Certes: please see my second comment above, about the merge templates, including an example of affected page. fgnievinski (talk) 00:14, 3 March 2022 (UTC)[reply]
I've seen all the hatnote templates mentioned above affecting non-portal excerpts, e.g., Surface#In mathematics. fgnievinski (talk) 00:17, 3 March 2022 (UTC)[reply]
Yes, they should be excluded from articles too. I maintain Module:Excerpt/portals but can't do much about non-portal excerpts, as changing them requires template editor privilege and I haven't studied the code in detail. Certes (talk) 00:20, 3 March 2022 (UTC)[reply]
@Certes: ah, ok, sorry for bothering you; I've created an edit request at Module_talk:Excerpt#Template-protected_edit_request_on_3_March_2022. fgnievinski (talk) 00:43, 3 March 2022 (UTC)[reply]
@Fgnievinski Done. Sophivorus (talk) 22:33, 3 March 2022 (UTC)[reply]
@Sophivorus: thank you! fgnievinski (talk) 23:31, 3 March 2022 (UTC)[reply]

Further suggestions of templates to blacklist

@Sophivorus: Further suggestions: ((Split)), ((Split section)), ((Split sections)), ((Split portions)), ((Split section portions)), ((Very long)). fgnievinski (talk) 23:56, 10 April 2022 (UTC)[reply]

And ((POV)) -- again, maybe all of Category:Hatnote_templates? fgnievinski (talk) 00:29, 11 April 2022 (UTC)[reply]

And ((more citations needed)) (plus its many redirects). fgnievinski (talk) 03:34, 11 April 2022 (UTC)[reply]

@Fgnievinski Hi! I just added Template:POV, Template:Split & redirects. As to the others, if you prepare the regexes, I'll be happy to add them. Sophivorus (talk) 13:44, 12 April 2022 (UTC)[reply]
@Sophivorus: here's for ((more citations needed)): '[Mm]ore citations needed', '[Aa]dd references', '[Aa]dditional citations for verification', '[Aa]dditional citations', '[Aa]dditionalcitations', '[Aa]ddref', '[Aa]dref', '[Cc]itations missing', '[Cc]itations needed', '[Cc]itationsneeded', '[Cc]ite sources', '[Cc]leanup cite', '[Cc]leanup-cite', '[Cc]leanup-verify', '[Ff]ew references', '[Ff]ew refs', '[Ff]ew sources', '[Ff]ewreferences', '[Ff]ewrefs', '[Ii]mprove references', '[Ii]mprove refs', '[Ii]mprove sources', '[Ii]mprove-references', '[Ii]mprove-refs', '[Ii]mproveref', '[Ii]mprovereferences', '[Ii]mproverefs', '[Mm]CN', '[Mm]ORECN', '[Mm]cn', '[Mm]issing citations', '[Mm]ore CN', '[Mm]ore Citations Needed', '[Mm]ore citations', '[Mm]ore cn', '[Mm]ore references needed', '[Mm]ore references', '[Mm]ore refs', '[Mm]ore sources needed', '[Mm]ore sources', '[Mm]oreCN', '[Mm]orecn', '[Mm]oreref', '[Mm]orereferences', '[Mm]orerefs', '[Mm]oresources', '[Nn]eeds additional citations', '[Nn]eeds more citations', '[Nn]eeds more references', '[Nn]eeds more sources', '[Nn]ot verified', '[Nn]otverified', '[Rr]ef Improve', '[Rr]ef improve', '[Rr]ef-improve', '[Rr]efImprove', '[Rr]eference improve', '[Rr]eferenced', '[Rr]efim', '[Rr]efimprove article', '[Rr]efimprove', '[Rr]efimproved', '[Rr]ip', '[Ss]ources needed', '[Ss]ources', '[Uu]nderreferenced', '[Vv]erification', '[Vv]erify', '\+R', '\+r', fgnievinski (talk) 15:29, 12 April 2022 (UTC)[reply]
@Sophivorus: and here's for ((POV)) '[Nn]pov', '[Pp]OV check', '[Pp]ov', '[Pp]ov check', '[Nn]POV', '[Nn]eutrality', '[Pp]OV-check-section', '[Pp]oint Of View', '[Pp]OV Check', '[Nn]POV Check', '[Nn]POV check', '[Pp]ov-Check', '[Pp]OV-Check', '[Pp]ov-check', '[Pp]ovcheck', '[Pp]OVCheck', '[Pp]OVcheck', '[Pp]ov Check', '[Pp]oV', '[Nn]eutral', '[Pp]ov-check-section', '[Nn]POV-dispute', '[Pp]OV dispute', '[Tt]oo friendly', '[Ww]hite washed', '[Pp]ov problem', '[Ss]pin', '[Nn]ot neutral', '[Pp]OV-check', '[Nn]PoV', '[Pp]oint of view', fgnievinski (talk) 15:56, 12 April 2022 (UTC)[reply]
@Fgnievinski Done, thanks! BTW, to avoid unnecessary performance loss, I think from now on it'd be better to blacklist templates as needed, and not preemptively. What do you think? Cheers! Sophivorus (talk) 17:43, 12 April 2022 (UTC)[reply]
@Sophivorus: thank you! all cases that I've reported I had encoutered while using ((excerpt)) in practice. maybe avoid the redirects? although regex usually is cheap. fgnievinski (talk) 17:52, 12 April 2022 (UTC)[reply]
@Fgnievinski Ah, I didn't know regexes were cheap, good to know. In any case, I think avoiding redirects is generally a good practice. Many redirects exist which don't have a single use throughout Wikipedia. Sophivorus (talk) 18:39, 12 April 2022 (UTC)[reply]

Further suggestions of templates to blacklist (2)

@Sophivorus: here's another one, please: '.*[Tt]opic.*' (used here). and may I ask if the existing '.*[Tt]imeline' shouldn't be generalized as '.*[Tt]imeline.*'? fgnievinski (talk) 00:14, 13 April 2022 (UTC)[reply]

@Fgnievinski Done, both! Thanks! Sophivorus (talk) 20:32, 13 April 2022 (UTC)[reply]

@Sophivorus: here's another one, please: '.*[G]lobal.*' (template:globalize). Thanks. fgnievinski (talk) 19:29, 11 May 2022 (UTC)[reply]

@Fgnievinski Done! Sophivorus (talk) 13:42, 12 May 2022 (UTC)[reply]

Possible bug?

I noticed this page has a Cite error (edit: it got updated with a workaround, error shows up here). It seems to use a ((Transclude lead excerpt)) that transcludes text from Jack Ma which has a ((Efn Chinese name)) that is being included. I would expect that efn to not be included (which would get rid of the error) so I thought I'd mention it here so that this could be addressed as it might potentially be a bug.

I apologize if this is the incorrect place to mention this. I just noticed the error and found my way here by clicking the talk page on ((Transclude lead excerpt)). Satricious (talk) 04:01, 9 February 2022 (UTC)[reply]

I'd like to note that this affects ((Efn Chinese name)) but not ((Efn)) as demonstrated by what I have set up on my sandbox here: User:Satricious/sandbox. Hope that helps. Satricious (talk) 04:43, 9 February 2022 (UTC)[reply]
@Satricious: Thanks for the report. I've fixed this in Module:Excerpt/portals, which will solve the problem in Portal: namespace, but fixing article namespace will need someone to amend Module:Excerpt. Certes (talk) 01:23, 10 February 2022 (UTC)[reply]
Great! The original error (in the portal namespace) is fixed. Cheers! Satricious (talk) 01:30, 10 February 2022 (UTC)[reply]

Is this the intended behavior?

I'll keep it brief. Please look at my sandbox as of right now and notice how when transcluding, some text is missing. Also notice how invoking the Excerpt module directly does not cause the text to go missing. Sorry for the brief report, but it's the best I can do.

EDIT: More context: The text that's not transcluded is ((As of|2015)). Is this intentionally blacklisted? Because it might cause issues like I mentioned above.

EDIT 2: Yeah I just looked at Module:Excerpt/portals and it's there. this is not a bug, my bad. But should it really be ignored? Maybe the article transcluded is using a poor use of the template?. Satricious (talk) 02:46, 10 February 2022 (UTC)[reply]

((As of)) produces useful text but also has side-effects which, whilst useful in the main article, may be unwanted in the extract. In particular, it adds the page to a dated maintenance category. Fortunately that only happens in mainspace, so use in portals should be OK, but it might be problematic when one article transcludes another. I'm not sure what we can do short of modifying such templates to accept a |extract=yes parameter or forking extract versions of them. Certes (talk) 11:41, 10 February 2022 (UTC)[reply]
Ah, so I see. Yeah I encountered this in the article namespace when I saw a section in an article that simply included a main link to another article. I figured it might be useful if at least the lead section of the linked article was included in the section so I attempted to transclude but that didn't work out as expected. Anyway, this doesn't seem to be much of an issue given the side-effects it would cause. Thanks for your response! Satricious (talk) 20:55, 10 February 2022 (UTC)[reply]

Citation error when transcluding from page with empty note list

I noticed today that Portal:1990s had a cite error (visible in this version). So the portal has a ((Transclude lead excerpt)) that transcludes from 1990s. Curiously, when I looked at that page I noticed it had an empty notes section which I removed (in this diff). And as you can see, the cite error is gone. I do not know if this error is related to the transclusion but I figured I'd post it here anyway. In any case this is probably a low-priority thing since most articles do not have empty note lists. Satricious (talk) 13:36, 26 February 2022 (UTC)[reply]

The error is intermittent; I saw it once in the old version but haven't been able to reproduce it to debug. The module removes templates such as ((efn)) but it's possible that some variant has slipped through. Unusually, that portal transcludes just an image and its caption with no text from the lead, then presents its own text with references. There are no references or notes in the image caption, nor were any removed recently. The intermittent nature suggests that the note is in an excerpt from another article which is occasionally transcluded elsewhere on the page at random. Please let us know if you see this problem again, so we may be able to identify the variant template and exclude it. Certes (talk) 14:13, 26 February 2022 (UTC)[reply]
I see, that really is some odd behavior. And sure, I occasionally browse Category:Pages with missing references list so if I see behavior like this again I'll happily let you guys know.
You mentioned not being able to reproduce this, I tried doing some naive testing in my sandbox and could not reproduce this either. However, I temporarily reverted my edits in 1990s (see page history if you wish), did not see any difference in the portal, purged, saw the error pop up, removed the note list, error persisted, disappeared after purging again (I know it's obvious changes should occur after purging but I'm just mentioning this as it might be useful to you).
This definitely has to do with the empty notelist. But it's really peculiar. Though I guess there is a rather unconventional dirty way of reproducing it though. Satricious (talk) 15:23, 26 February 2022 (UTC)[reply]
I think the footnote comes from ((Airreg|N|485US|,)) in USAir Flight 405 which appears randomly as selected article 5. The template isn't excluded because it produces useful text as well as an unwelcome footnote. The symptoms resemble ((Nihongo foot)), a related disease which we cured by implementing a simplified version of that template in Module:Excerpt/portals. Airreg is more complicated, as the format depends on the first parameter: for N (U.S.) we just concatenate the parameters, but some country prefixes add a hyphen or other decoration. Certes (talk) 18:49, 26 February 2022 (UTC)[reply]
I've changed Module:Excerpt/portals to remove the footnote from Airreg. This change does not affect Module:Excerpt but should fix the portal. Certes (talk) 19:22, 26 February 2022 (UTC)[reply]
Oh wow, so I was completely wrong. I'm impressed that you managed to nail down exactly what the issue was so quickly. Props to you! I appreciate all the work you guys do to get things like this working :) Satricious (talk) 21:04, 26 February 2022 (UTC)[reply]

Template-protected edit request on 3 March 2022

In line 4 of Module:Excerpt/config, where it is: 

<code>
	blacklist = {
</code>

please change to: 

<code>
	blacklist = {
		'[Ff]urther', '[Ff]urther2', 
		'[Oo]utline', 
		'[Rr]efimprove',
		'[Ss]pecial[Cc]hars',
		'[Mm]erge from','[Mm]erge to',
<code>

Discussion: Module_talk:Excerpt#Suggestions_of_templates_to_blacklist. Thanks! fgnievinski (talk) 00:42, 3 March 2022 (UTC) fgnievinski (talk) 00:42, 3 March 2022 (UTC)[reply]

Citation error: The time allocated for running scripts has expired.

I have used ((excerpt)) twice in this page and the moment I used the second (under Telecommunications section), the error:

The time allocated for running scripts has expired.

replaced several references. There are also errors in several other pages that use ((excerpt)). Can someone please identify the cause and fix the error? —Sanglahi86 (talk) 18:52, 6 March 2022 (UTC)[reply]

It appears this template is very expensive. I temporarily removed the second excerpt from the page. Checking the revision with the excerpt mysteriously does not show the errors, although the other pages still contain the errors.—Sanglahi86 (talk) 19:09, 6 March 2022 (UTC)[reply]
I've checked the first few "other pages" linked above and it seems the error message only appears in search results, the actual articles seem fine. fgnievinski (talk) 03:43, 11 April 2022 (UTC)[reply]

Cite error: Invalid <ref> tag

Hi, thanks for ((excerpt)), I love it. I'm getting the following errors in my sandbox:

It seems to involve a named reference ("O Globo"), defined in the transcluded section, which ended up renamed to prefix the transcluded article title. Could you please take a look or give me some pointers to debug it? Thanks! fgnievinski (talk) 00:38, 4 April 2022 (UTC)[reply]

@Fgnievinski:, it appears ((excerpt)) is very expensive. Using ((#lsth:pagename|sectionX)) might be a better option. I tried using ((#lsth:Política externa do governo Jair Bolsonaro|França)) on that page and the references show up correctly. Although I can't find an option of it supporting paragraph transclusion like what excerpt does. —Sanglahi86 (talk) 13:30, 10 April 2022 (UTC)[reply]
The module seems to be removing ((Citar web)) templates, leaving empty citations. It may possibly be connected to a bug at pt:Módulo:Excerpt/config#L-24, where the square bracket should end after Mm instead of enclosing the whole word. (The second pattern incorrectly matches "c".) Certes (talk) 14:29, 10 April 2022 (UTC)[reply]
@Certes: Brilliant, that was it -- fixed now, many thanks! fgnievinski (talk) 23:14, 10 April 2022 (UTC)[reply]

Sidebar templates

Module:Excerpt had been changed to blacklist sidebars back in 2018. But some sidebars still appear in excerpts, please see test2. fgnievinski (talk) 04:12, 11 April 2022 (UTC)[reply]

@Fgnievinski Hi again! If you visit Module:Excerpt/config you'll notice that templates are blacklisted based on their name, so unfortunately it's not possible to simply blacklist all sidebars once and for all. I just added the ones in your test page to the blacklist though. Cheers! Sophivorus (talk) 13:58, 12 April 2022 (UTC)[reply]
@Sophivorus: ah, I see; I thought it was a bug regress. thank you for the wildcards, I think they can cover the most common instances. but here's an exception: Template:Antisemitism sidebar is not being caught in the '.*[Ss]idebar.*' regex. and the 'WWII timeline' could be generalized as .*[Tt]imeline.*', see here for a broader test. thanks! fgnievinski (talk) 16:10, 12 April 2022 (UTC)[reply]
@Fgnievinski Done! Sophivorus (talk) 17:43, 12 April 2022 (UTC)[reply]
@Sophivorus: But but... you didn't fix '.*[Ss]idebar.*', you've hard-coded '[Aa]ntisemitism'; that way, I'll have to bug you every time I come across a new sidebar. I thought a better approach would be to make '.*[Ss]idebar.*' match 'Antisemitism sidebar', no? Similarly for '.*[Tt]imeline.*', which is not catching 'WWII timeline' in here. fgnievinski (talk) 17:55, 12 April 2022 (UTC)[reply]
@Fgnievinski '.*[Ss]idebar.*' does match 'Antisemitism sidebar', but The Holocaust calls 'Antisemitism' rather than 'Antisemitism sidebar', so I had to blacklist it exlplicitly. As to the WWII timeline, it is matched by '.*[Tt]imeline.*', but it shows in your test page because you call it explicitly at User:Fgnievinski/test#Start and end dates. Sophivorus (talk) 18:38, 12 April 2022 (UTC)[reply]
@Sophivorus: ah, you're right, my bad! I didn't realize sidebar might have redirects, too (for exameple). what a nightmare! I wonder what would be a sustainable long-term solution? blacklisting every sidebar manually seems a lot of work. fgnievinski (talk) 19:34, 12 April 2022 (UTC)[reply]
would it be feasible to blacklist a category of templates, such as Category:Hatnote templates? fgnievinski (talk) 19:34, 12 April 2022 (UTC)[reply]
would it be feasible to follow redirects so that we need to blacklist only non-redirect templates? fgnievinski (talk) 19:34, 12 April 2022 (UTC)[reply]
@Fgnievinski Yes it's quite an issue, but luckily it's only as serious as excerpts are popular. Blacklisting by category or auto-detecting redirects is possible in principle, but probably very expensive. I'm thinking a more reasonable solution may be to blacklist only non-redirect templates (and perhaps a few exceptions for very high-use redirects), and then document the limitation and ask users to either change the template they want to exclude to its cannonical name, or blacklist it explicitly using templates=-Antisemitism, for example. Sophivorus (talk) 20:19, 12 April 2022 (UTC)[reply]

Images in sidebar templates

it seems images included in sidebar templates are excerpted -- is that intentional? although it works well in most cases, sometimes the image is not labeled (for example, ((History of Japan))), which looks odd. other times, it doesn't work at all, see here. fgnievinski (talk) 16:15, 12 April 2022 (UTC)[reply]

@Fgnievinski The thing is, because not all infoboxes are named according to a pattern, I use the presence of a .jpg or .png file to determine which template is the infobox and extract the image. In this case, the sidebar got picked, but it can be fixed adding files=0, cheers! Sophivorus (talk) 17:45, 12 April 2022 (UTC)[reply]
Understood, will use "files=0" when necessary. fgnievinski (talk) 19:52, 12 April 2022 (UTC)[reply]
In the case of Second Italo-Ethiopian War, it seems the problem is caused by ((multiple images)), see here. fgnievinski (talk) 19:56, 12 April 2022 (UTC)[reply]

Undefined referenes through excerpt

I'm trying to sort out why 59th Venice Biennale gives errors about the reference "Artnews" being undefined. The text (and the reference definition) are brought in by ((excerpt)) from List of national pavilions at the 59th Venice Biennale. The source artilce renders without error, and I've done some poking at it to try to figure out what's wrong ... but nothing makes a fix.

Anyone able to help me sort it out please? - Mikeblas (talk) 22:38, 25 April 2022 (UTC)[reply]

References from quoted articles have the article name prepended, in this case <ref name="List of national pavilions at the 59th Venice Biennale Artnews">. This is to prevent clashes when both articles use the same name (commonly <ref name=":0"> etc.) for different citations. Certes (talk) 23:28, 25 April 2022 (UTC)[reply]
Is that relevant here? I don't think it is becuawe 59th Venice Biennale doesn't invoke any references defined in the excerpted text. -- Mikeblas (talk) 01:06, 26 April 2022 (UTC)[reply]
Partly. The problem seems to be that some of the citations use just name="Artnews" but others use name="Artnews" group="". The name isn't being changed when it appears with group="". Pinging Sophivorus. Certes (talk) 09:59, 26 April 2022 (UTC)[reply]
I must be missing something; all the usage I see is <ref name="Artnews"/>, without a group name attribute (not even a blank one). -- Mikeblas (talk) 13:50, 26 April 2022 (UTC)[reply]
No, I was missing something. I put ((excerpt|List of national pavilions at the 59th Venice Biennale|National pavilions)) through Special:ExpandTemplates and noted that some ref tags had group="" and didn't get a prefix. I hadn't spotted that they were expansions of ((r)) in the original article. That no longer happens now that the article contains a workaround. Certes (talk) 14:44, 26 April 2022 (UTC)[reply]
Observation: it seems to work fine when it came across more named references, such as <ref name="Artnews"/>, but fails when it is trying to call the reference via ((r|...)) on the remote article. I replaced one instance of r with the declaration and the problem went away. For your immediate need @Mikeblas: you could do the rest of those, as in Special:Diff/1084771632. — xaosflux Talk 14:05, 26 April 2022 (UTC)[reply]
That works! Sorry, I thought I had already removed uses of the ((r)) tempplate -- except for the invocations which were commented away. So it looks like the problem is that ((excerpt)) is simply incompatible with ((r)). -- Mikeblas (talk) 14:10, 26 April 2022 (UTC)[reply]

Template-protected edit request on 17 May 2022 - Rare section edge-cases

Note: This is actually a dual edit request, with the pages in question being both Module:Excerpt and Module:Transcluder. Im only going to do 1 edit request to avoid polluting anything.

This is a potential fix for the transcluder and excerpt not correctly recognising a section's name if it contained a #. Note that while the testcases for excerpt have shown this change should be perfectly fine, I would preferably like a comment from anyone who knows more about this module than me to mention if there was a reason for ([^#]*) over (.*) (if any) before a TE or other implements this, as it feels odd enough to potentially be intentional.

The relevant fixes are on the sandbox for each page. The main excerpt testcases can be found here and the examples of it successfully working with the # are visible on my test page. Note that /testcases3 is claiming that there are differences between the 2 versions on some examples. After using safesubst to get the exact content and using a comparison tool, no difference was noted. Im not entirely sure what has happened there.

Thanks. Aidan9382 (talk) 21:17, 17 May 2022 (UTC)[reply]

@Aidan9382 Done, thanks for the fix and careful request! BTW, the issue with /testcases3 has been there for a long time and seems caused by Template:Test case table Sophivorus (talk) 21:15, 18 May 2022 (UTC)[reply]
Thanks for implementing! That testcases bug feels very odd to be happening, but thanks for letting me know. Might take a look at it, who knows. Aidan9382 (talk) 04:31, 19 May 2022 (UTC)[reply]
@Sophivorus: Thanks for bringing that whole /testcases3 issue has been happening for a while thing. Been working on that module for the past hour or so, and it really needs some slight bug fixing. The UNIQ--QINU detection was the issue here (fixed now), but the nowiki and nowiki+ output options are apparently not doing so good and making the diff checker think this isnt right (Maybe its somehow skipping normalisation?), so i guess ive got something to focus on now. Aidan9382 (talk) 05:32, 19 May 2022 (UTC)[reply]
@Aidan9382 Awesome, I didn't know you were working on that module! The testcases look green already, so thanks, nice work! Sophivorus (talk) 11:26, 19 May 2022 (UTC)[reply]

Recent change?

Has there been a recent change to the module - I'm noticing pages that previously worked are once again getting the error "The time allocated for running scripts has expired" --Find bruce (talk) 22:52, 28 May 2022 (UTC)[reply]

Always link to an example page when you are reporting a possible problem. – Jonesey95 (talk) 23:50, 28 May 2022 (UTC)[reply]
Thanks for the reminder. An example is Electoral results for the district of Bathurst, but the problem exists in virtually every article in Category:New South Wales state electoral results by district. You will note that every election prior to 1930 uses transcluded sections rather than the template. This was done in December 2021 [1] to deal with the error. Find bruce (talk) 02:21, 29 May 2022 (UTC)[reply]
@Find bruce: This issue has existed for a while - ive noticed quite a few election articles that have had this error for a while. As for recent changes, no. This module and its main associated module (Module:Transcluder) have had no significant changes that would affect performance. Pages can just sometimes take longer or shorter to load each time you save a page with Excerpt. Hope this helps. Aidan9382 (talk) 03:58, 29 May 2022 (UTC)[reply]
Thanks @Aidan9382: - the issue never arose when I first started using the module in June 2020 & first appeared around August last year. I edited all 350 articles to implement a work around. I will wait to see if anyone has a solution, otherwise I will simply give up on the module & go back to my previous template approach for all but the most recent elections - it's not as elegant & there are traps for the unwary that the module avoids, but it works with fewer resources. Find bruce (talk) 05:48, 29 May 2022 (UTC)[reply]
@Find bruce: There seem to have been quiet a few changes to the Transcluder last year august, but I don't see any differences that could be major. If your having to implement a fix for 350 pages, then I think it might be time this module set had a performance look-over, as that could cause a bit of confusion for passing editors. I'll take a look, see if I can find any performance improvements, but for now, I guess the template approach will have to do. Aidan9382 (talk) 05:59, 29 May 2022 (UTC)[reply]

@Find bruce: - Just came across an interesting discovery. It turns out that one of the main expensive calls (as in a third of the entire lua runtime) is to do with it trying its hardest to find files. Of course, in this situation, files are not gonna be relevant (at least I doubt it). Adding |files=0 to all the excerpts on a page should significantly improve the performance, and infact should be good enough to remove lua timeout errors in 99% (or 100% if we are lucky enough) of scenarios. Give it a shot, tell me how it goes. Aidan9382 (talk) 07:45, 29 May 2022 (UTC)[reply]

I haven't investigated the current code but it may be worth looking at Transcluder.removeNonFreeFiles(). A previous version of the module (now forked for use in portals) spent a lot of its time opening every file's rationale to check whether it could be included, and there was no obvious way to do that more efficiently. Certes (talk) 18:28, 29 May 2022 (UTC)[reply]
@Certes: While I could easily see that becoming a quick problem (just the sound of it makes me concerned), that unfortunately isn't the issue here. Transcluder.removeNonFreeFiles is only ran if any files are found. The issue here is the fact its going to extents much further than whats worth it to attempt to find files unless explicity told not to, and that is leading to another expensive Transcluder.get call. I will note that there may be reason for concern, as I've just noticed the code only attempts to remove Non-free files if found in blacklisted templates (Not a common scenario. I can explain further if required). Do you think this is worth looking into as another potential issue with this module? Thanks! Aidan9382 (talk) 18:36, 29 May 2022 (UTC)[reply]
@Aidan9382: I think the "blacklisting" is just another way to specify files, e.g. files=-2 means all files except the second, and behaves just like files=1,3-999. The check looks unchanged from the portal version except that it's also checking Commons, which may be unnecessary as Commons doesn't accept non-free content. It would be nice if we had an inexpensive isFileNonFree method, but sadly we do need to read the text; people got very upset and started deleting transclusions when we didn't. However, as you say, this may be a red herring as that code only runs when a file is found. Certes (talk) 19:38, 29 May 2022 (UTC)[reply]
Thanks @Aidan9382: I have added |files=0 to all the excerpts in Category:New South Wales state electoral results by district & it appears to have fixed all of the current timeout issues. Find bruce (talk) 08:56, 30 May 2022 (UTC)[reply]

Expensive calls and why excerpt runs so slow

During some recent testing, I was giving a prod at some of the parts of excerpt, and I noticed a section to do with desperately trying to grab files from anything we might have missed if a file cant be found initially (Lines 152-173). After excluding the content during some testing and some show preview usage, I noticed the execution time dropped by about 42% (Average 7.4438s -> Average 4.304s). As expected, the actually troubling part of that is probably the second Transcluder.get() for just templates. However, I am running very short on solutions, so I was wondering: Could someone give their opinions on:

I have a solution idea in mind, however the elegance of it is very low and is quite a hacky idea that would probably be a bad move, so input from others would be nice. Aidan9382 (talk) 18:26, 29 May 2022 (UTC)[reply]

@Aidan9382 Amazing find!! The usefulness of the feature is basically that many articles (perhaps even most) have their main image in the infobox, so this code tries to salvage it. It's inefficient, messy and even unreliable, I agree, but also useful and perhaps even expected at this point. That being said, I think I have an idea on how to fix it! I tried to implement it here but I couldn't quite finish (it's late and I can't find a good way to debug!). Basically the idea is to avoid entirely the second call to get() which essentially fires a second request and is probably the main cause of the performance loss. @Certes The call to removeNonFreeFiles also fires a request and may be causing a performance loss, so I'm thinking that given that all the trouble you had was in portals, we can maybe disable this feature on the mainspace by default, and only enable it on a case-by-case basis when required, thus avoiding the performance loss. Sounds like a plan? Sophivorus (talk) 21:26, 30 May 2022 (UTC)[reply]
@Sophivorus: Disabling the check in mainspace is worth a try. We need the current behaviour (in Module:Excerpt/portals, not Module:Excerpt) for portals because many of them display an excerpt from one article picked randomly from a long list, and no one checks them all to see if one has a blurry movie still that will bankrupt Hollywood if displayed. Certes (talk) 21:38, 30 May 2022 (UTC)[reply]
@Aidan9382 @Certes I think I nailed it (see Module:Excerpt/sandbox). I still have to fix a few minor issues, but would you mind checking if the effort was worth it, Aidan9382? (I have no idea know how to benchmark Lua modules.) Thanks! Sophivorus (talk) 23:23, 30 May 2022 (UTC)[reply]
@Sophivorus: I don't know the current code well enough to review it properly but that change to checking freefiles looks like what you need. (I'm interpreting that flag as "limit to free files only?".) For a quick benchmark, I preview with a relevant page, view the source code (Ctrl-U on Firefox) and search for "NewPP limit report". That shows how close we are to various limits (rarely a problem except in portals) and gives timings for various templates and Lua functions. Certes (talk) 23:36, 30 May 2022 (UTC)[reply]
@Certes Awesome! Well, judging by the "NewPP limit report" for Electoral results for the district of Bathurst it seems the sandbox version is far superior (~ 6 secs vs ~10 secs)! I'll wait for Aidan while I fix the remaining issues and deploy in a day or two, cheers!! Sophivorus (talk) 00:12, 31 May 2022 (UTC)[reply]
@Sophivorus: I've gone ahead and done some comparisons between Except and Excerpt/sandbox on a test page, and I can confirm the /sandbox version fixes the issue quite well, bringing my test page from an average of 8s to an average of 6s (The results are being really inconsistent so the improve is probably better than 1/4th). My only concern with doing Transcluder.getTemplates(excerpt) is that, if the excerpt is set to only=files (or similar), the excerpt will include no templates, and therefore wont find any infoboxes, but this is more of a rare scenario. While you could technically grab an unfiltered excerpt of the section and filter out what the user wants later, that also kinda nullifies a significant part of Transcluder's get(), so I'm wary of trying to implement that. Other than that though, this looks like its doing a fine job. Nice work! Aidan9382 (talk) 05:06, 31 May 2022 (UTC)[reply]
My theory seems to be correct, but it also seems to not be removing non-free files. Just take a look at the differences from Template:Excerpt/testcases. Also check Template:Excerpt/testcases2 for a weird template difference at the bottom. Note: If the content side by side looks the same, but its marked as different (yellow, uncollapsed), then its a formatting difference. Ignore it, consider it the same. It probably is. Aidan9382 (talk) 13:23, 31 May 2022 (UTC)[reply]
@Aidan9382 Thanks for the insights! I was able to fix the issue at the bottom of /testcases2, here. As to the non-free files, as I mentioned to Certes above, it's just a change in the defaults. From here on non-free files will have to be filtered explicitly by setting freefiles=yes. This is because 99% of excerpts don't have issues with non-free files but each check for non-free files is expensive. Finally, regarding the issue in /testcases where only=files no longer gets files from the infobox, I agree this is an edge case not worth the trouble. If no more issues or concerns arise, I'll deploy in a day or two, cheers! Sophivorus (talk) 22:38, 31 May 2022 (UTC)[reply]
@Aidan9382 @Certes Deployed! I'll stay around for some minutes just in case, thanks for the awesome work and support! @Find bruce This new version should make setting files=0 unnecessary. Thanks for bringing up this issue in the first place! Sophivorus (talk) 22:07, 1 June 2022 (UTC)[reply]

Template-protected edit request on 29 May 2022 - Blacklist fix

In the /config, protection padlock (pp) templates are considered blacklisted. However, the current regex, [Pp]p%-.+, catches all protection templates except the core ((pp)) one. To fix this, change 

'[Pp]p%-.+',  -- ((pp-move-indef)) etc.

to 

'[Pp]p', '[Pp]p%-.+',  -- ((pp-move-indef)) etc.

The reason its over 2 different regexs is because simply doing [Pp]p.* might be a bit too aggressive, and its easier to stay on the safe side of things. Thanks. Aidan9382 (talk) 21:18, 29 May 2022 (UTC)[reply]

 Done. P.I. Ellsworth , ed. put'r there 00:18, 30 May 2022 (UTC)[reply]

Underscore ref name bug

Agriculture in California is using an excerpt of California wine which produces Cite error: The named reference California wine Misson_Grape was invoked but never defined (see the help page). Examination of the resulting HTML suggests excerpt does not understand _. Invasive Spices (talk) 2 June 2022 (UTC)

Ah, this isn't an issue due to the underscore. This error is because the reference is defined in the infobox (| wine years = ((years ago|1769))<ref name="Misson_Grape">((cite web| url=https://www.wine-blog.org/index.php/2011/03/17/the-mission-grape-played-a-major-role-in-california-wine-viticultural-history/ | title=The Mission Grape Played a Major Role in California Wine Viticultural History | author1=Diaz, Jo |website=Wine Blog | date=2011-03-17))</ref>), and is only a short named reference in the text that was actually excerpted (<ref name="Misson_Grape"/>). For a solution, I'd recommend defining it in the text and keeping it as a short named reference in the infobox. Aidan9382 (talk) 15:57, 2 June 2022 (UTC)[reply]
I've implemented the fix, so don't worry about it. Aidan9382 (talk) 16:03, 2 June 2022 (UTC)[reply]

Appears to exclude template

Excerpt appears to be excluding ((Election box 2pp)) and I'm not sure why. The template adds a row "Two-party-preferred result" to the election results table - see for example Results of the 2022 Australian federal election in Victoria#Aston which is excerpted to Electoral results for the Division of Aston#2022, where everything but that template. I tried setting templates=.* however that then excerpted only that template & nothing else --Find bruce (talk) 05:05, 4 June 2022 (UTC)[reply]

Huh, thats odd. I don't immediately see it in the template blacklist. I'll get looking into it in a short moment. Aidan9382 (talk) 05:08, 4 June 2022 (UTC)[reply]
@Find bruce: Thanks for the notice - this seems to be a potential bug with the module. I'll go ahead and look into a fix. For now, just dont worry about getting the 2pp in - I don't see a hacky workaround here. Aidan9382 (talk) 05:23, 4 June 2022 (UTC)[reply]

@Sophivorus: (or others) I've implemented a fix for the situation above (It's an issue with how the blacklisting / whitelisting is done and how either its all blacklist or no blacklist, which is a bit inconvenient) that should mean |templates= now correctly overwrites the blacklist. However, I've done this in a way that, in my opinion, might be a bit too off brand, and I'd like input from someone else before submitting an edit request to get this fixed. (This involves a change to both Module:Excerpt/sandbox and Module:Transcluder/sandbox). Opinions? Aidan9382 (talk) 05:59, 4 June 2022 (UTC)[reply]

@Aidan9382: We discussed some ideas in #Short description being transcluded above. Certes (talk) 09:35, 4 June 2022 (UTC)[reply]
@Certes: Thanks, I missed that. From reading, it seems a decent solution here would be to simply append the "templates" parameter to the default blacklist if it starts with hyphen (-) and replace it if it doesn't. After all, I can't recall of any actual cases where someone wanted to "exclude templates X and Y but include the ones in the blacklist". Do you? It's always "exclude templates X and Y as well as those in the blacklist". I'll try recreate this on the sandbox and see how it functions. If it fixes the idea, I'd be much happier to put that up for an edit request. Aidan9382 (talk) 09:43, 4 June 2022 (UTC)[reply]
@Certes: Not only was that super simple to implement, but it seems to have worked fine, and should still follow what Template:Excerpt says for the expected functionallity. If that looks all good to you, I'll put up a TE edit request. Aidan9382 (talk) 09:48, 4 June 2022 (UTC)[reply]
@Aidan9382: That looks good to me and even works with the edge case List of United States tornadoes in 1950, but best for Sophivorus to have a quick look at it before it goes live. Certes (talk) 09:57, 4 June 2022 (UTC)[reply]
Alright, cool. May I ask, what was the edge case at Tornadoes 1950? Oh and, please reply with direct source - I have no idea whats going on, but Bawl! is freaking out and replacing the previous section with a duplicate (Just see your edit diff in history and itll make sense). Aidan9382 (talk) 10:00, 4 June 2022 (UTC)[reply]
Not sure what went on with Bawl there, but thanks for fixing. Tornadoes now uses |templates=.* but had previously used |templates=-Coords to include all templates except Coords (a template not used here, not to be confused with Coord which is used and is included). Certes (talk) 10:09, 4 June 2022 (UTC)[reply]
Certes, Aidan9382, a few things happened. When you add a comment, you really do a section edit. Obtain the original text for the section, insert comment into text, perform section edit with new section text. Bawl edited the wrong section: section 20 instead of 21, so the previous section was overwritten with the text of this section plus your comment. This was an odd combination of circumstances. There are several ways to get the right section number: either just take it from the section edit link, search for the section title or.. the other way. When the section number is taken from the section edit link, it'll break if archiverbot passes by while you are reading or if someone adds a subsection in a section above the one you're commenting on. That issue affects all editors. Well, all but Bawl.
Bawl uses the section title when editing a whole section because there isn't much else. (a section could be empty after all) The section #Cite error: Invalid <ref> tag uses nowiki tags in the title. As Bawl can only get the section title from the HTML, any codes can cause issues. To some degree this gets compensated, wikilinks in a title for example are no problem. Nowiki on the other hand.. Using &lt;ref&gt; should work better, but that section can't be edited as a section currently.
But when commenting it actually follows a different strategy. It becomes completely agnostic towards the section title. Section title changed while your were reading? No problem. Bawl determines the section number itself. In this process, various elements are escaped to avoid counting "sections" that are actually code examples or otherwise not actually sections. That ref tag above was an issue as it was seen as an unclosed ref tag. Another issue is found in #Template-protected edit request on 3 March 2022 where within pre tags two opening code tags are found. The second one was meant to be a closing code tag and neither of them work because they are within pre tags. Assuming I won't duplicate this section, I fixed this issue: Bawl now escapes pre and nowiki first as those could legitimately contain unbalanced tags which would otherwise throw off the escape process. — Alexis Jazz (talk or ping me) 12:50, 4 June 2022 (UTC)[reply]
Categories