Additionally, NewsPlex can be used to quickly explore large lists of candidate news-servers and evaluate their service value. This is called the Minimal mode.
NewsPlex runs on your own machine (or possibly on some other machine) and appears as a standard news-server to your news-reader and as a news-reader to your news-servers. It is therefore somewhat similar to a proxy or to a web cache.
NewsPlex can be controlled through command-line options at startup time, or dynamically, from a Web browser or by telnetting into it. The provided NpNNTP command line utility can be used to issue commands to NewsPlex, for example from shell scripts.
NewsPlex runs on Windows (95/98/Me/NT/2000/XP), OS/2 and Unix (Linux, Solaris and FreeBSD). This version and this manual are for Unix.
Handling of news-groups is deferred. All news-groups are originally in the inactive state. This allows to reduce disk, cpu and network usage and was a prerequisite for handling the full news-group list. The activation of a given news-group is performed automatically, when a news-reader accesses it, ie. requests the list of articles it contains. Up to a few thousand news-groups can be activated before performances drop.
The first time a news-group is entered, NewsPlex generates an internal article announcing that the news-group is now being downloaded, starts downloading the news-group from all news-servers, waits for a few seconds to have a chance of getting some real articles, and returns the resulting list to the news-reader. The full news-group will be available a few moments later.
NewsPlex is plug and play. If run without any configuration or command-line options, it will auto-configure itself to handle the full list of news-groups, and will use all the news-servers which it can find on your local network or referenced in files on your machine. It will complete this list with a few well-known specialized news-servers.
It is later possible to add or remove news-servers. It is also possible to restrict the group list, for example to save memory. It is also possible to improve performance by either tuning some parameters or activating features like the asynchronous article retrieval.
NewsPlex can service several news-readers at the same time, which notably allows to control it from the network while it is already being used by some news-reader.
NewsPlex constantly monitors whether the news-reader which has requested an article is still there to receive it and has not disconnected in the meantime. If not, it stops the retrieval tentatives immediately.
NewsPlex can be seamlessly inserted between your existing news-server
and your news-reader, without disturbing the list of news-groups or
the article numbers.
This only works if you do not use the auto-configuration.
Note: this feature is currently broken.
NewsPlex always uses the best news-server for requesting an article, among those having it. It will use the remaining news-servers if the best one could not deliver the article. Servers are rated dynamically according to current speed, best speed obtained in the past, latency and availability.
NewsPlex remembers errors occurring in the middle of an article retrieval, typically news-server timeouts or unexpected connection terminations, and will try to use a different news-server as source the next time the article is requested.
NewsPlex can be run in the background, permanently. It will explore news-servers periodically to maintain its article database up to date.
NewsPlex does not need to be stopped if the network connection goes down, for example because you disconnect from your ISP. It will wait until the connection is up again and continue operations.
NewsPlex is able to use any news-server as news source, even news-servers which implement very old versions of the news transfer protocol, and yet always exports a modern interface to news-readers. Errors occurring during the exploration of news-servers are also recovered, through either restarts, deferring or workarounds. article descriptions imported from news-servers are carefully checked for correctness. Those which appear garbled, typically a result of overruns in the news-server, will be asked for again, as long as progress is made. In non-recoverable situations, bad article descriptions are ignored and not stored in the article database; a correct version will most probably be available from some other news-server. NewsPlex is able by itself to fix incorrect or missing Bytes, Lines and Message-ID information for articles (in the last case, a Message-ID of the time-stamp@newsplex or similar form is generated). Such errors do not cause rejection of article descriptions.
NewsPlex supports connecting to news-servers either directly, or through proxy servers (SOCKS 5, SOCKS 4, HTTPS, simple tunnels or Sun-specific).
NewsPlex has simple spam filtering possibilities based on the From, Message-ID and Subject fields of article descriptions. Wild-cards are supported.
NewsPlex allows optional login/password authentication of users, and offers two levels of access: ordinary/user and privileged/superuser/administrator.
NewsPlex can be used as a client of another NewsPlex. (It can even be a client of itself, the internal synchronization scheme allows this.)
NewsPlex automatically removes junk lines from the headers of posted articles. Through extra header lines, news-readers tend to disclose more information about your software and hardware configuration than what you usually want other people to know, and this cannot generally be disabled by other means. Currently, the Sender, User-Agent, X-Accept-Language, X-Binary-Poster, X-Mailer, X-MimeOLE, X-MSMail-Priority, X-Newsposter, X-Newsreader, X-Posting-Agent, X-Priority, and X-WM-Posted-At lines are eliminated. NewsPlex can of course do nothing about header fields added by the news-servers themselves.
asynchronous article retrieval allows to accelerate retrievals by parallelizing operations and eliminating idle moments.
An arbitrary number of article retrievals can be scheduled, and up to 4 retrievals are performed at once (this can be adjusted with the -a command-line option). NewsPlex remembers which articles remain to be retrieved even across shutdowns and restarts and finishes retrievals the next time it is run.
You are not obliged to access the asynchronously retrieved articles through the async news-group, because asynchronously retrieved articles are also visible as async/xxxxxxxx.ok files. If articles contain binary attachments, you can directly run arbitrary decoding tools on the files and even delete them when done.
| Since version 3.6, the NpUud uudecoding utility is provided and since version 3.7 it also supports the yEnc format, which notably offers reliable decoding and automatic multipart merging. NpUud can extract the binary attachments from any file and put whatever remains into a separate .hdr file named after the first attachment. |
If your news-servers are unreliable, become easily unreachable or give timeouts, NewsPlex will isolate you from these errors, perform the necessary retries in the background and keep you posted with the latest news in the minimal time and without drudgery.
Few news-readers allow using several news-servers at the same time, and very few can take articles from several news-servers for a single news-group transparently, or request an article from different news-servers until successful. NewsPlex allows you to choose your news-reader on other criteria than these abilities.
Because NewsPlex can service many users at the same time, running on a well-known machine on your local network, it can avoid everybody the work to setup and maintain a multi-server configuration themselves and can globally reduce the amount of data transferred with the Internet.
NewsPlex version 3.9 is freeware. It is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. You can use it at your own risk, and report whatever problems you have encountered with it. You are free to copy this software ONLY if you include this document file and all the other accompanying files with it. You may NOT charge anyone for a copy of this software other than a small copying fee. You may NOT include this software with any commercial software without the written consent of the author.
Unpack the NewsPlex distribution archive to any directory. You should notably have the executable file newsplex and a help file newsplex.htm in HTML format, which you are currently reading. There should also be a boilerplate file file_id.diz and a descript.ion file with descriptions of all the files in the distribution. There are also several utilities provided, which can assist you with dealing with binary and text files of various types.
Because of the auto-configuration feature, you can start NewsPlex without providing any command-line options or configuration files. However, if you do not want to rely on that feature, you need to provide one subdirectory and two files in it to start. They must be located either in the directory from which NewsPlex is started or in the directory indicated through the -d command-line option. This directory becomes the work directory.
| Filename | Description |
|---|---|
| config/servers |
This text file contains the list of news-servers to use
and identifies the main one.
It can also specify which proxy, if any, should be
used for accessing a given news-server (see a specific chapter below).
A line describing a news-server has the following format: server-numberserver-url [flags] [# comment] Example:
1 news://news1.yourisp.com # server without login
2 news://mylogin:mypass@news2.yourisp.com m # server with login (the main server)
3 news3.yourisp.com:4000 # server on port 4000
# this is a comment
server-number must be unique for each news-server and greater than 0. It identifies the news-server in the article database and cannot be changed after a news-server has been explored. The second field should be the URL (Uniform Resource Locator) of the news-server, that is its Web address. It must contain at least the hostname of the news-server, and can optionally contain the news:// prefix, the login name, the associated password and the port of the news-server. A given login name, hostname and port combination can be provided only once.
If you need to pass funny characters in the
login or in the password, use standard % escape
sequences: a % character followed by the hexadecimal
ASCII number of the offending character. For example, @
can be escaped as %40, : as %3A
and a blank as %20. So if the login is
for example ano:nymous and the password
joe@domain.com, the resulting URL will be
Flags are explained in a separate table below. It is possible to add comment lines, starting with a #, or to put comments at the end of lines, starting them with #. A line containing just the word exit will make NewsPlex ignore the rest of the file. A line containing just the word dontexplore makes NewsPlex not automatically explore the news-servers which follow. |
| config/groups.yes |
This file contains the names of news-groups which
should be retrieved from news-servers and offered to
news-readers. The names of news-groups can contain *
and ? wild-cards. Eg:
*midi*
*music*
It is possible to pass a single * on a line alone. This will select all news-groups on all news-servers and is the default auto-configuration value. Around 100 bytes of memory or swap are necessary to store information about a single news-group, so selecting all news-groups will use about 10 megs. |
| Flag | Meaning |
|---|---|
| A |
Keep-alive the news-server for 15 minutes
after it was last used. Normally, connections to
idle news-servers are closed after 5 minutes, without
keep-alive.
This flag can be automatically set by NewsPlex if the news-server appears as difficult to connect to, for example because it limits the number of concurrent users and is very popular at the same time. It is also set if a news-server closes the connection immediately after a TCP connect. The NPCONNECTED command and the Web page ignore the keep-alive when displaying the duration of idleness for news-servers. |
| a |
Attempt retrieval of articles even if the news-server
does not have the news-group or the requested articles in that
news-group. Every time an article is requested, NewsPlex
will attempt to retrieve it from the news-server as well.
Some server have articles which are not or which are no more referenced in any of their news-groups. It is still possible to retrieve them by Message-ID. NewsPlex normally attempts this during zombie retries. Passing this option allows to do it for specific news-servers all the time. |
| B | Issue a GROUP request before an article retrieval by Message-ID if the group where the article is located is known. This is normally unnecessary, but might help with some news-servers. |
| C | Consider that errors occurring when NewsPlex is listing available news-groups on the news-server indicate the end of list. This allows to use otherwise unusable news-servers. |
| cvalue |
Define a cost for news-server. The value will be considered
as the number of kilo-bytes per second to
subtract from the news-server 's score when evaluating it
in order to retrieve an article.
Having the value expressed in kilo-bytes per second
rather than in bytes allows to keep the display width
of the flags field not too large.
Note that the cost notion only applies after the category notion, which means it is only used among news-servers of the same category. |
| D | Do not reuse the news-server for an asynchronous article retrieval after it becomes free. Setting this flag for news-servers which do not allow many simultaneous connections should give the news-reader less 500 No server having article is reachable errors if the news-reader tries to retrieve some articles directly when asynchronous article retrievals are enabled. |
| E | Some news-servers do not like that news-readers request the article descriptions for empty groups, and start providing corrupted article descriptions from this moment. This option avoids this. |
| F |
If the news-server fails to provide descriptions for
articles which he has previously declared as
available, consider these articles
as permanently unavailable.
If in the news-server exploration logs you see that for certain news-servers NewsPlex keeps exploring certain articles ranges without getting any article descriptions, you might set this flag. This however introduces a penalty: if the news-server generates article descriptions out of order and at the same time declares the numbers in advance, you might not get all the article descriptions. Note that when this option was introduced, NewsPlex did not yet implement the collapsing of neighboring article number ranges, so today it is less interesting. |
| G | Do not use the XGTITLE request for getting the list of groups. This flag is equivalent to the -x command-line option except it only applies to one news-server. |
| g(group-name1 group-name2...) | Do not explore specific news-groups on the news-server. Set this option if accessing a certain news-group causes malfunction of the news-server, like disconnects or indefinite waits. |
| H | Use HEAD requests rather than XOVER requests to obtain the article descriptions from the news-server. NewsPlex normally does this automatically when necessary, but you can force it, for example if XOVERs are slow. See also the x flag. |
| I | Allow certain article description truncations to occur after the Message-ID field and still accept the article description. NewsPlex only accepts cuts which do not look like a corruption which could be recovered by asking for the article description again, and cuts in the middle of article threading references, since some news-servers limit article descriptions to an arbitrary length and this cut-off falls there. |
| i | Ignore login failures. NewsPlex will attempt to use a news-server even if the login sequence failed. |
| L | When constructing the list of interesting news-groups offered by the news-server, request the full list and filter it locally instead of requesting only the names of news-groups matching wild-cards. |
| M | Do not try to get articles by Message-ID from this news-server. NewsPlex tries to retrieve an article by Message-ID if the article cannot be retrieved by number from its news-group, typically because it has been deleted from it. However, some news-servers stop responding in such circumstances, presumably because the search for the article in other news-groups takes a lot of time. |
| m | This news-server is the main news-server (the notion is explained below). Only one news-server can be main. |
| N | This flag can be displayed, but cannot actually be set. The news-server most probably does not allow posting, at least this is what the welcome message says. |
| n | Blind login. Do not wait for the news-server welcome message and issue a command immediately, typically an article download request. This flag is useful with certain news-servers only, which issue the welcome message with a delay, and in the meantime accept other commands. |
| O | LISTGROUP does not set the current news-group. This flag allows to deal with news-servers which do not adjust the current news-group when receiving the LISTGROUP news-group request. This is not so easy to detect automatically, hence this flag. |
| o(proxy-url1 proxy-url2...) | This flag allows to specify a list of per-news-server proxies, which do not need to (and also even cannot) be explicitly given a proxy id. Use this option if you do not want to manage the numbering of proxies yourself. |
| P | This flag can be displayed, but cannot actually be set. The news-server probably allows posting, at least this is what the welcome message says. |
| R |
Increase the maximum number of connections allowed
to the news-server by 2. This flag can be repeated, in order
to increase the limit more. It can be useful with
news-servers which limit the speed but do not
limit the number of simultaneous connections.
Note that you will also need to increase the maximum number of simultaneous asynchronous article retrievals if you increase this per-news-server setting beyond that value, otherwise it will not be effective. |
| r |
Reduce the maximum number of connections allowed
to the news-server by 1. This flag can be repeated in order
to lower the limit even more. It can be useful with
news-servers for which the default limit of 4
simultaneous connections is too high, for example
because their owners charge you per connection count
or per connection time.
Note that repeating this flag too many times will prevent NewsPlex from using the news-server at all. |
| S | The news-server is slow, so do not use the LISTGROUP or XHDR LINES requests for getting the article numbers, but rely on the results on the GROUP request instead. |
| T |
Give the news-server much more time to answer requests. This
allows to avoid disconnects with news-server which take a lot
of time to answer some requests, and hence for example
it is not possible to complete an exploration successfully.
Note that if a news-server is given more time, NewsPlex will also wait longer before switching to another news-server which could provide the same article. |
| tcategory |
Set the category of the news-server, ie. its download priority, to
value category, which can be negative, 0
or positive. Zero is the default value.
This allows to enforce the
order in which news-servers are used for downloading articles.
NewsPlex will not use a news-server with a higher category until it has
tried to get the article from news-servers of lower categories.
Note that NewsPlex will not wait for a lower category news-server to become available if all allowed connections to this news-server are already used. (Of course, it will attempt to open another connection to a lower category news-server, if the quota is not yet reached, even if there is already an open and available connection to a higher category news-server.) NewsPlex also opens connections to each news-server one by one, to avoid flooding a news-server with too many simultaneous connection attempts, so it can switch to a higher category news-server if a connection attempt is currently pending for a lower category news-server. Finally, you should set the D flag (Do not reuse the news-server for an asynchronous article retrieval after it becomes free) for news-servers for the categories to be fully effective during async downloads. |
| U |
Do not explore the news-server automatically.
The news-server will not be explored periodically,
it will also not be explored after a news-group has been
activated and it will not be explored after an article
has been posted to it (assuming it is also "main").
The operator can still request a single exploration
with NPEXPLORESERVER or thru the Web
interface.
This flag can be interestingly combined with the a flag, to create a news-server where NewsPlex tries to take articles from blindly, by Message-ID, without ever asking for anything else. |
| X | Give extra time to news-server for responding when connecting. Some overloaded news-servers may need this. Use this flag if the news-server can be normally connected at TCP level, but subsequently does not send a welcome message before NewsPlex stops waiting and considers the connection as failed. Only use this flag on good news-servers, because it increases the time before NewsPlex tries to use another news-server and hence may slow down things actually. The flag can be repeated. |
| x | Use XHDR requests rather than XOVER requests to obtain the article descriptions from the news-server. You can set this mode of operation if XOVERs are slow. See also the H flag. |
| ydays | Limit the age of article descriptions added from this news-server to the article database to days days. By default, the age is not limited. This flag takes precedence over the -y command-line option. See also the NPDELETEOLD command. |
The main news-server is used for posting articles by default. Additionally, the NPPOSTSERVER command can be used to change the posting news-server globally at any time, or to view the current selection.
If you do not select any news-server as main, NewsPlex will consider all the news-servers as equivalent, explore them simultaneously, and number articles starting from 1 up.
Three further files are optional:
| Filename | Description |
|---|---|
|
config/groups.no
(optional) |
This file indicates the names of unwanted news-groups
which might get selected by config/groups.yes.
No wild-cards are allowed here. Eg:
alt.music.boring
alt.midi.lame
Use this file only if you want to forbid access
to specific news-groups.
|
|
config/authinfo
(optional) |
This file is used if the -A command-line option is passed on
the command line. It can define the login names and passwords
which should be used for protecting access to NewsPlex.
It should contain one or two lines of the following form:
user usernamepassword
supervisor usernamepassword
The user line is equivalent to passing the -U
command-line option on the command line, and the supervisor line is
equivalent to passing the -P command-line option. The
last occurrence of a given line type is retained.
|
|
config/banner
(optional) |
This file is used if present, and can define a different welcome message for NewsPlex. Only the first line from the file is used. Unlike the other files from the config directory, this file is reread at each incoming connection. |
|
config/filters
(optional) |
This file is used for spam filtering,
with the NPFILTER
command and with the -f command-line option.
The format is as follows: all lines starting with a # and all empty lines are ignored. Other lines must be composed of: a pattern for the From field, followed by a tabulation and by a pattern for the Message-ID field of articles to be deleted, optionally followed by a tabulation and a pattern for the Subject field. Wild-cards (*, ? and character sets and intervals, eg. [0-9a-z] or [!;,]) can be used in all patterns. Example file: # this is a comment annoying@annoying.com <*@annoying.com> # this is another comment lamer*@lamers.com <lamers*@*lamers.com> # filter out reposts * * REPOST:* |
If you modify any of these files (except config/banner), you will need to restart NewsPlex before the modifications are taken into account. The restart can be performed from the Actions web page, with the NPRESTART command or by shutting down NewsPlex and re-running it.
To shutdown NewsPlex, send it the SIGINT signal either from the keyboard if NewsPlex runs in foreground, using the Ctrl-C or Del keys, or with the kill command. Sending a second SIGINT signal or any other signal will terminate NewsPlex unconditionally.
It is also possible to stop NewsPlex from the Web browser interface or by telnetting into it (telnet localhost 119) and entering the NPSHUT command. This command can also be sent with the NpNNTP utility.
The simplest way of defining proxies is to do this individually for each news-server, using the o(proxy-url1 proxy-url2...) per-news-server flag.
Otherwise, you can define proxies globally. This allows to use the same proxy for several news-servers, at the inconvenience of having to give them unique numbers.
In this second case, first, define a list of proxies you intend to use, with entries like this:
proxy proxy-url,proxy-group-number [# comment]
The proxy-url must start with the proxy type, either none, or socks5://, or socks5b://, or socks4://, or socks4b://, or https://, or tunnel://, or sun://.
none means that no proxy should be used and the connection can be established directly. This is the default way of connecting to news-servers.
If the type is not none, the proxy-url should then specify the name of the proxy server and potentially its port as well as the login and the password if necessary. The default ports are:
| Port | Proxy types |
|---|---|
| 1080 |
socks5 socks5b socks4 socks4b |
| 443 | https |
| 3666 | Sun |
tunnel proxies are proxies which only allow to access a single machine, and which connect to it automatically after being connected to. Since they appear just like normal news-servers, they can be specified instead of news-servers in the config/servers file. However, if a given news-server can be accessed through several such proxies, it is interesting to define them as tunnel proxies instead, because this will make NewsPlex try them in sequence when connecting to the news-server. Note that unless one of the proxies specified is not none, the original news-server will not be contacted directly anymore, so its name will not matter. Tunnel proxy URLs have the default port equal to the port of the URL to which they are used to connect.
The proxy-group-number must be a small integer number. It allows to identify a proxy and to reference it from a news-server entry. The default proxy-group-number is 0.
Note: socks5b servers are buggy socks5 servers and socks4b servers are buggy socks4 servers. If you observe that NewsPlex is unable to connect at NNTP level to any news-server, and the logs/results file shows truncated welcome messages from them, with missing first 6 chars (eg. instead of 200 newsservername... you see wsservername..., and instead of 502 You are not allowed to talk you see u are not allowed to talk, use socks5b instead of socks5 and socks4b instead of socks4 to define the type of your proxy.
For each news-server entry, it is possible to reference the proxy in the server-url by appending a final ,proxy-group-number to it. By default, value 0 is assumed. To quickly modify a config/servers file used on a direct Internet connection so that it can be used on a proxy-ized Intranet connection, just add an initial line defining proxy 0. Example:
proxy socks5://login:pass@socksproxy:1080,0 proxy none,1 proxy sun://oursunproxy,2 1 news1.yourisp.com,0 # server outside, will go through proxy 0 2 mylogin:mypass@companyserver,1 # server on local network, no proxy 3 news3.yourisp.com:4000,0 # another outside server, use proxy 0 4 news4.someisp.com,2 # go through sun proxy 2 to reach thisNote that all ,0 are actually unnecessary.
If no proxy is defined in proxy group 0, NewsPlex will automatically assume a direct connection for this proxy group, that is, create an implicit proxy none,0 entry. This allow to not define any proxies in the config/servers file if the connection is always direct.
NewsPlex will use the first proxy defined in a given proxy group, except if the news-server indicates that it does not allow additional connections from this proxy (or from your host, if the proxy is none).
In such a case, NewsPlex will try to use the next proxy from the same proxy group, so that the news-server thinks a different news-reader is connecting. NewsPlex will also try to use a different proxy from the same proxy group if the current proxy indicates that it does not authorize establishing the NNTP connection. Finally, NewsPlex will try to use another proxy if the first one could not be connected to. NewsPlex will not try to use another proxy if the news-server is simply not reachable from the first proxy in a proxy group.
To deal with this situation, NewsPlex supports proxy chains. This means that it can use a proxy to reach another proxy, and then use this second proxy to actually connect to a news-server.
After proxies have been defined in the config/servers file, you can chain them using the proxychain directive:
proxy socks5://intranetproxy,0 proxy socks5://externalproxy,1 proxychain 0 1 1 news.somedomain.com,0Here, proxy group 1 is chained behind proxy group 0. Every connection going through proxy group 0 will transit through proxy group 1 before reaching the news-server. Referencing proxy group 1 directly remains possible.
Only two components are allowed in proxy chains. A single proxy group can appear only once in the first position of a proxy chain.
If the second proxy group has several proxies, only the first one defined will be used. However, all proxies in the first group will be tried if necessary.
Proxy chains are longer to establish and slower in operation than ordinary proxy-based connections, which themselves are worse than direct connections. They also require that the first proxy does not forbid connecting to the second one. In the above example, intranetproxy must allow connecting to port 1080 on externalproxy.
If a zombie article is requested, NewsPlex behaves like for requests for articles residing on unreachable news-servers. It returns the same error code, but with a specific message (500 Article is a zombie) by default. If renumbering is activated, NewsPlex pretends that the article has been deleted (which is actually true here) and renumbers it, so that if the article becomes available again, it is perceived as a new article by the news-reader.
It is also possible to specify the a flag for specific news-servers, which will make NewsPlex attempt to always retrieve articles from them, even if they never said they had the article or even the news-group.
| Filename | Description | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| dataset2 |
The dataset2 directory contains these parts of article descriptions which do not
need to be loaded into memory all the time during operations.
There is one file for each news-group offered by NewsPlex. Each file
has one line per article. It describes the article
and has the following fields:
|
||||||||||||
| index |
The index directory contains these parts of article descriptions which need to
be loaded into memory during operations. There is one file for
each news-group offered by NewsPlex. Each file has one line per
article. It describes the article and has the following fields:
|
||||||||||||
| async | The async directory is used when asynchronous article retrieval is enabled. It stores both control files and the articles themselves. | ||||||||||||
| etc/scheduld |
The etc/scheduld file contains the list of articles which
are being retrieved asynchronously.
It is deleted, together with the
etc/done file, after all articles have been processed.
Each line in the etc/scheduld file has the following fields:
|
||||||||||||
| etc/done |
The etc/done file contains the list of asynchronously retrieved articles
which have been completely processed.
It is deleted, together with the
etc/scheduld file, after all articles have been processed.
Lines in the etc/done file have the same format as lines
in the etc/scheduld file, except more status codes are
possible:
|
||||||||||||
| logs/summary |
The logs/summary file stores the log of asynchronous article retrievals.
Both successful and unsuccessful attempts
are logged there after they have completed. For a given
article, there can be several failed logs before the
successful one. Or the last log can be a permanent
failure, eg. article deletion from the article database.
If it grows over 500000L bytes, a new file is started, and the existing file name is added the .bak extension. The previous .bak file is deleted. |
||||||||||||
| async/xxxxxxxx.ok |
The async/xxxxxxxx.ok files, where x are hexadecimal
digits, are asynchronously retrieved articles.
They are also visible in the async news-group generated
by NewsPlex locally. They are deleted 300
seconds after having been successfully
retrieved for the first time by a news-reader. If they are
requested again in the meantime, the deletion is delayed
accordingly.
You can also delete them manually at any time if you want.
This allows to retrieve articles several times, which
is useful when news-readers do not cache them, and also allows to
better deal with errors (eg. disk space exhaustions)
occurring when transferring multi-part postings. news-readers might
discard parts already retrieved if an error
occurs for the current part, even if for NewsPlex
they were retrieved correctly and hence scheduled for
deletion.
Note that any async/xxxxxxxx.ok file, where x are lowercase hexadecimal digits and where the first digit is not a zero, copied into the async directory, will be visible in the async news-group and can be retrieved from a news-reader. After the first successful retrieval, it will be scheduled for deletion as described above. |
||||||||||||
| async/xxxxxxxx.del | The async/xxxxxxxx.del files, where x are hexadecimal digits, are produced instead of async/xxxxxxxx.ok files if an article could not be retrieved, for example because it has been deleted from all news-servers. async/xxxxxxxx.del files contain only an error message from NewsPlex, formatted as a news article. These files are deleted just like async/xxxxxxxx.ok files after a news-reader has requested them. You can delete them manually at any time if you want. | ||||||||||||
| async/xxxxxxxx.log | The async/xxxxxxxx.log files, where x are hexadecimal digits, are logs of asynchronous article retrievals. They exist only while the article retrieval is in progress and when it completes, they are appended to the logs/summary file. | ||||||||||||
| async/xxxxxxxx.tmp | The async/xxxxxxxx.tmp files, where x are hexadecimal digits, are asynchronously retrieved articles in progress. After the article retrieval is finished, articles are renamed to async/xxxxxxxx.ok and visible in the async news-group generated by NewsPlex. | ||||||||||||
| async/xxxxxxxx.bad | The async/xxxxxxxx.bad files, where x are hexadecimal digits, are articles which could not be retrieved completely because of temporary errors. These files always have a size greater than zero, otherwise they are not kept. During the next article retrieval attempt, they will be deleted and replaced with async/xxxxxxxx.ok files. | ||||||||||||
| etc | The etc directory hosts the various output files that NewsPlex produces and sometimes reuses. | ||||||||||||
| etc/groups |
The etc/groups file contains the full list of
news-groups managed by NewsPlex. Each line
records information about a single news-group,
and contains a unique news-group number,
the date at which the news-group has been added
to the database, and the name of the news-group.
You can currently (as of version 3.9) delete this file when NewsPlex is not running, in order to get rid of news-group names which do not exist on any news-server anymore, yet match the content of the config/groups.yes file (news-groups which do not match the current config/groups.yes files are ignored when NewsPlex reads this file). NewsPlex will reconstruct the file when it is run again. This only introduces a minor penalty that the names of all news-groups will be returned if later a news-reader asks for new news-groups. |
||||||||||||
| etc/popular |
The etc/popular file records news-group popularity, which is measured
by the total number and size of articles retrieved from them.
When exploring a news-server, NewsPlex first asks for the most popular news-groups, but it also takes into account the last exploration date for each news-group. This avoids starvations with unpopular news-groups if a news-server often becomes unavailable before an exploration completes.
Each line of the etc/popular file contains
the number of articles requested, the total size in kilobytes
of these articles, the date of last article retrieval
expressed in seconds since 1/1/1970 and the name of the news-group.
|
||||||||||||
| etc/retrievd | If the -R command-line option is passed, with at least the l flag, NewsPlex stores in the etc/retrievd file the Message-ID fields of all the articles which have been successfully retrieved by news-readers. This file is also used when the -M command-line option is specified (see section below). | ||||||||||||
| explore |
The explore directory stores the log files resulting from the
exploration of news-servers. Log files are named after news-servers;
see the description for the newsrc2 directory for format
details.
Each news-server exploration has the same scenario, which is logged into the file:
|
||||||||||||
| journal |
The journal file logs certain events and conditions as NewsPlex runs,
like starts of news-server explorations and incoming connections
from news-readers. This file is renamed to journal.bak each
time NewsPlex is started and this file is over 204800L
bytes long. The previous journal.bak, if any, is deleted.
When NewsPlex terminates, it prints into the journal file some summary information about the usage of news-servers during the session, similar to the results of the NPSERVERS command. |
||||||||||||
| list | The list directory temporarily stores the full news-group listings from news-servers which cannot return news-groups matching patterns. It is automatically destroyed when NewsPlex exits. | ||||||||||||
| logs | The logs directory hosts the various output files that NewsPlex produces but does not reuse. Its content can be deleted at any time. | ||||||||||||
| logs/cli-host-port |
The logs/cli-host-port files log commands issued by your news-reader
and the responses it got.
They notably contain the full content of posted
articles.
host is either the name of the machine from where the news-reader connected, or the IP address if the name could not be found, while port is the IP port number the news-reader is using. Once a news-reader has disconnected, the contents of its logs/cli-host-port file are appended to logs/clients and the logs/cli-host-port file is deleted. |
||||||||||||
| logs/web-host-port | The logs/web-host-port files are similar to logs/cli-host-port files, except they are used for Web browser sessions. Once a Web browser has disconnected, the contents of its logs/web-host-port file are appended to logs/web and the logs/web-host-port file is deleted. | ||||||||||||
| logs/clients | The logs/clients file logs closed sessions from news-readers. It can be deleted at any time. If it grows over 100000L bytes, a new file is started, and the existing file name is added the .bak extension. The previous .bak file is deleted. | ||||||||||||
| logs/compact | The logs/compact file logs database compacting activity. A compaction is scheduled to start 10 minutes after NewsPlex startup, and is performed every 300 minutes later. The logs/compact file can be deleted at any time. If it grows over 10000 bytes, a new file is opened when the next compaction starts, and the existing file name is added the .bak extension. The previous .bak file is deleted. | ||||||||||||
| logs/results |
The logs/results file indicates the exploration
results for news-servers. Each line has this format:
server-url [flags] # result (comment)
This file is kept when NewsPlex terminates. It can be deleted
at any time.
|
||||||||||||
| logs/unusable | The logs/unusable file is only created in Minimal mode (-m command-line option). It contains the logs of news-server explorations for all news-servers which did not have any interesting news-groups or which could not be contacted at all. It can be deleted at any time. | ||||||||||||
| logs/intruder | The logs/intruder file logs rejected incoming connections. For each connection, the date, the IP address, the IP port and the host-name are stored. The file can be deleted at any time, but you better check from time to time who tries to use your NewsPlex without authorization. | ||||||||||||
| logs/web | The logs/web file has the same purpose as the logs/clients file, except it collects the logs of past Web sessions. | ||||||||||||
| newsrc2 |
The newsrc2 directory contains one file per news-server used.
It logs which active news-groups have been explored on the
news-server and which article descriptions have been retrieved
from them. The file does not store information
about non-active news-groups on the news-server.
The names of files are normally equal to news-server names, but if there is a login or a non-standard port specified for a news-server, the server-disk-name will include them as well, using the [login@]server-name[_port] syntax, so that it is possible to eg. use several news-servers located on the same host. Note that invalid characters in the resulting filename are replaced with underscores. The newsrc2 directory is not called newsrc, like in versions of NewsPlex older than 3.0, so that your article databases are not corrupted if by accident you run a version of NewsPlex older than 3.0 in the directory containing 3.0 and later style article database files. It will simply create and start filling the database and newsrc directories, which you can safely delete after stopping the program. Then you can start the correct version of NewsPlex. The format of the files in the newsrc2 directory and the information in them are not quite the same as for the newsrc or .newsrc files used by news-readers. Lines starting with a tab contain the name of a news-group and the date of its last exploration, while the other lines contain the ranges of article numbers for which article descriptions have already been retrieved. If the news-group is empty on the news-server, there will be no numbers for it. You can delete a file corresponding to a news-server to force NewsPlex to retrieve the full article list for each news-group at the next exploration of the news-server. This will however almost certainly create dangling references in the article database. Such dangling references can be removed with the NPCHECKGROUPS command, preferably after the news-server has been re-explored. |
||||||||||||
| telnet.htm | The telnet.htm file contains a Web redirection to NewsPlex 's telnet port. By opening it with a Web browser, you should get a telnet session with NewsPlex. It is created at startup and deleted at shutdown. JavaScript must be enabled in the Web browser. | ||||||||||||
| telnet.url | The telnet.url file contains an Internet Explorer link to NewsPlex 's telnet port. By opening it with Internet Explorer, you should get a telnet session with NewsPlex. It is created at startup and deleted at shutdown. | ||||||||||||
| news.htm |
The news.htm file contains a Web redirection to NewsPlex 's
news-server. By opening it with a Web browser, you should
get your news-reader started and connecting to NewsPlex.
It is created at startup
and deleted at shutdown. JavaScript must be enabled in
the Web browser.
Note that the login and password are not currently passed in the news.htm link. |
||||||||||||
| news.url |
The news.url file contains an Internet Explorer link to NewsPlex 's
news-server. By opening it with Internet Explorer, you should
get a news-reader session with NewsPlex. It is created at startup
and deleted at shutdown.
Note: some news-readers, like Outlook Express, do not recognize the port specification in the URL, so if you run NewsPlex on a non-default port, you will need some tweaking to get them to connect. Note also that the login and password are not currently passed in the news.url link. |
||||||||||||
| web.htm | The web.htm file contains a Web redirection to NewsPlex 's Web page. By opening it with a Web browser, you should get to NewsPlex 's web page. It is created at startup and deleted at shutdown. JavaScript must be enabled in the Web browser. | ||||||||||||
| web.url | The web.url file contains an Internet Explorer link to NewsPlex 's Web page. By opening it with Internet Explorer, you should get to NewsPlex 's web page. It is created at startup and deleted at shutdown. | ||||||||||||
| panic | The panic file is created if NewsPlex detects an non-recoverable internal error. Once a short message has been logged into this file, NewsPlex exits. A core file is produced. | ||||||||||||
| update | The update directory contains updates to index files. NewsPlex now avoids rewriting index files each time they are modified and instead appends new article descriptions to them and stores updates to existing article descriptions into update files. When update files grow too large, or if the index files are small, index files are rewritten and the update files are deleted. | ||||||||||||
| xhdr | The xhdr directory temporarily stores the results of individual XHDR requests issued to news-servers for which the x flag has been set. Hosted files are named using the server-disk-name_xhdr-request-name pattern (see the description of newsrc2 directory for explanations about server-disk-names). They are deleted when no more needed. The directory is automatically destroyed when NewsPlex exits. |
Although the STAT command could be executed entirely locally, it actually goes to news-servers in order to allow validating the availability of an article.
The XHDR command has been extended.
NewsPlex also adds a few commands of its own, which are not intended for usage by news-readers but rather by an operator. These commands can be entered after performing telnet localhost 119 on your computer. They can also be performed using the provided NpNNTP utility. The HELP command does not display them if the user is unprivileged.
Conversely, it is possible to enable the Web browser interface in NewsPlex by passing the -w port command-line option, and to connect from any Web browser to url http://localhost or to url http://localhost:port if port was not 80.
| Command | Web | Description | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| NPACCESSED | Yes |
Display which news-groups are currently being accessed by
news-readers or are being updated.
The following information is displayed for each news-group:
|
||||||||
| NPACTIVE | Yes | Like LIST, but displays only active news-groups. | ||||||||
| NPAPPLYRETRIEVD | Yes | Read the etc/retrievd file and mark as read in all news-groups all articles whose Message-IDs are listed inside. Equivalent of the -M command-line option. | ||||||||
| NPASYNC [min-lines] | Yes | Enable asynchronous article retrieval for articles longer than min-lines lines and which seem to contain binaries (NewsPlex looks at article subjects to decide about that). If min-lines is preceded with a minus, all articles longer than min-lines are retrieved asynchronously. Passing min-lines of 0 disables asynchronous article retrieval, but does not stop article retrievals already scheduled. The NPASYNC command also displays the list of articles which are being retrieved asynchronously, followed by a line giving their count and total number of bytes. | ||||||||
| NPCANCEL article-number | Yes | Cancel the retrieval of an article scheduled for asynchronous article retrieval. The article-number can be provided either in decimal form or in hexadecimal form (with a 0x prefix). It is not necessary to select the async news-group first. | ||||||||
| NPCHECKGROUPS mode | Yes |
Check the article database for coherency with the newsrc2
files and optionally remove dangling news-server references.
Such references can appear if NewsPlex is brutally killed
before having saved modifications done to an index file,
while corresponding modifications done to the
newsrc2 file have already been saved (the
opposite situation simply makes NewsPlex retrieve
some article descriptions again, as regets).
mode specifies the operation mode:
0 - check only, silent mode
If a news-server is being explored when this command is entered, no checking will be performed for it. Starting with version 3.0, NPCHECKGROUPS also deletes the index and dataset2 files corresponding to unwanted news-groups. |
||||||||
| NPCLIENTS | Yes |
Display the list of currently connected client news-readers and
Web browsers.
For each client, the following information is displayed:
|
||||||||
| NPCLOSE server-number | Yes | Close all idle connections to a given news-server. If no server-number is provided or if it is 0, all idle news-server connections are closed. | ||||||||
| NPCONNECTED | Yes |
Display which news-servers are currently being connected to.
This can be either for exploring or for retrieving an article
on news-reader request. Connections are maintained a few minutes
even when idle, so as to minimize the time necessary to
re-use a news-server.
For each connection, the following information is displayed:
|
||||||||
| NPDATASETS | Yes |
Display the usage information of the files in the
dataset2 directory. For each active dataset,
the following information is displayed:
The last displayed line shows:
|
||||||||
| NPDEACTIVATE | Yes |
Deactivate the current news-group.
The dataset2, index and update files will
be deleted, and NewsPlex will consider that
the group has never been activated.
Note: Because, for now, the newsrc2 files are cleaned off non-active news-groups at next exploration only, it is better to shutdown NewsPlex before re-activating a news-group, or the news-group will not be re-populated with news-server references correctly. |
||||||||
| NPDELETEOLD days | Yes |
Delete article descriptions older than days days
in all news-groups.
Articles which do not have a recognizable Date will
be kept. Note that NewsPlex does recognize some non-English
formats and can parse post-1999 dates where the year is noted
with one or two digits only, eg. 0, 00, 01, etc.
No zombies are created; article descriptions are deleted directly. Note that article descriptions are normally automatically removed from the database when corresponding articles vanish from news-servers (after the expiration of the zombie period). The NPDELETEOLD command allows to trim the article database in case news-servers forget to expire some of their old articles. See also the -y command-line option and the y per-news-server flag. |
||||||||
| NPDELETESERVER server-number | Yes |
This command currently deletes article database news-server references to the
specified news-server. It does not make NewsPlex stop attempting
to use the news-server.
If a news-server becomes unusable, as the explore/server-disk-name file indicates, you can delete it from NewsPlex using NPDELETESERVER server-number. NPGLOBALSTATS will then not show any articles for it anymore. Then you can delete it from the config/servers file and restart NewsPlex. This command even allows to delete a news-server which is no more present in the config/servers file. For example, if you removed the line, but did not perform NPDELETESERVER first. |
||||||||
| NPEXPLORESERVER server-number | Yes | Explore a news-server immediately rather than waiting till its time comes. This command is also useful if NewsPlex has been started with automatic exploration disabled (-u). Note that the news-server will be only explored once. | ||||||||
| NPFILTER | Yes |
Apply spam filters from config/filters to current news-group.
After selecting a news-group with the GROUP name
command, you can enter NPFILTER to apply
the current set of filters from the config/filters file
to this group.
No zombies are created; article descriptions are deleted directly. See also the -f command-line option. |
||||||||
| NPFORCETASK handle | Yes | Force the immediate execution of a task, which otherwise would be executed later. One can force an asynchronous article retrieval this way. | ||||||||
| NPGLOBALSTATS | Yes |
Display provider stats for all news-groups.
NewsPlex will display for each news-server how many articles it has and how many of them are unique. If a given news-server does not appear in the list, NewsPlex has no articles from it. |
||||||||
| NPGROUPSTATS | Yes | Display provider stats for the current news-group. After selecting a news-group with the GROUP name command, you can enter NPGROUPSTATS to see the news-servers which contain articles for this news-group. | ||||||||
| NPMARKRETRIEVED article-number | - | Mark article article-number in the current news-group as retrieved (see section below). | ||||||||
| NPPOSTSERVER [server-number] | Yes | Define a news-server as the posting news-server. This command can be used to dynamically change the news-server set by the config/servers file. This is a global setting, valid for the current and all other connections. Without argument, this command displays the current posting news-server. With argument 0, it disables posting. | ||||||||
| NPREFRESH news-group-name | Yes | Start exploring the indicated specific news-group on all news-servers. Use this command if you feel you need to refresh NewsPlex 's idea about it. Note that this command does not activate the news-group implicitly. | ||||||||
| NPRENUMBER level | Yes |
This command helps dealing with unavailable
news-servers (asynchronous article retrieval is another way).
If no news-server having an article is reachable, by
default NewsPlex will send to the news-reader
500 No server having article is reachable
Some news-readers however do not
like this message and do not request further articles,
or even disconnect.
If NPRENUMBER is activated, but is not at level 3, NewsPlex will instead send
423 no such article number in this news-group
which makes the news-reader think the article has simply been
deleted. Most news-readers will continue retrieving other
articles correctly.
NewsPlex will however create a new reference for the article which will be seen by the news-readers the next time they asks for new articles. There are 2 possible ways for creating this new reference:
It is possible to revert to the normal operation by sending NPRENUMBER 0. |
||||||||
| NPREWRITEGROUP | Yes | Integrate the content of the update file into the index file for the current group, and delete too old zombies. This command is useful for quickly trimming a news-group, notably after setting a shorter NPZOMBIEDAYS value. | ||||||||
| NPRESTART | Yes |
Shutdown NewsPlex and restart it immediately.
All configuration files will be re-read.
The original command-line options will remain active.
Once the NewsPlex welcome message is displayed
again, it is possible to enter further commands.
The previous context of the connection (current news-group,
current article, etc.) is lost however.
The following parameters will not be restored to
the original values:
|
||||||||
| NPSERVERS | Yes | Display information about the usage of news-servers. For each news-server, NewsPlex will display whether it has already taken article descriptions from it during the current session or was able to connect at all. Other information is the time since the last exploration, the current speed, the number of articles and bytes retrieved and the flags (as in the config/servers file documentation above). | ||||||||
| NPSHUT | Yes | Shutdown NewsPlex. NewsPlex will terminate. This command can take a while to execute, both because some operations are not easily interruptible, and because lots of data may need to be written to disk. | ||||||||
| NPTASKS | Yes |
Display the list of currently running and scheduled tasks.
A task is something to do asynchronously and not necessarily
immediately; tasks are categorized and only a limited
number of tasks in the same category are performed in parallel.
For each task are displayed:
|
||||||||
| NPZOMBIEDAYS [zombiedays] | Yes | Equivalent to the -z command-line option. Without argument, this command displays the current value only. |
| Option | Description |
|---|---|
| -a count | Perform count asynchronous article retrievals at once instead of default 4, if asynchronous article retrieval is enabled. This parameter should be equal to the speed of your Internet connection divided by the average speed of your news-servers. Passing 0 will disable the retrieval of articles scheduled for asynchronous article retrieval. |
| -g min-lines | Enable asynchronous article retrieval for articles longer than min-lines lines and which seem to contain binaries (NewsPlex looks at article subjects to decide about that). If min-lines is preceded with a minus, all articles longer than min-lines are retrieved asynchronously. |
| -L |
This option makes NewsPlex return an article deleted
status to the news-reader when an article retrieval is scheduled. If you do not
want to see the fake articles and/or do not want to have
to delete them, pass this option.
When setting this option, you probably should configure your news-reader to not automatically combine parts of multi-part postings (except for the async group), because news-readers will usually stop retrieving further parts if they encounter a missing part in the middle. Note that NewsPlex can properly deal with news-readers which request an article by Message-ID if they could not get it by number, and avoids scheduling the same article for asynchronous article retrieval twice. |
| -X program-name or -X "program-name args..." |
This option allows to define the name of the program
which should be run every time NewsPlex finishes
retrieving a
async/xxxxxxxx.tmp file in the async directory, but
before renaming it into async/xxxxxxxx.ok.
This program could for example be a uudecoder
or a MIME attachment extractor.
It can do whatever it wants with the async/xxxxxxxx.tmp file,
even delete it.
Notably the NpUud program now provided with NewsPlex
can be used.
The program will be given the full pathname of the async/xxxxxxxx.tmp file. It is possible to pass additional arguments to the program, by double-quoting the entire invocation line, so that it appears as a single argument to the option. NewsPlex can run several copies of the program in parallel. It is up to the tool to perform necessary synchronization. |
| -Z count | Allows to set how many zombie retries will be conducted at once. Default is 1. Greater values can speed up the retrieval of zombie articles, but at the cost of more open connections to news-servers. Value 0 will disable zombie retries. |
| Option | Description |
|---|---|
| -c | Delete empty files and directories and exit. |
| -C | Delete all created files and directories and exit. Use with caution. Only the config directory will be preserved. |
| Option | Description |
|---|---|
| -e | Enable insecure mode and allow remote connections. By default, only connections from the machine running NewsPlex are possible. |
| -p port | Use port port for establishing the NNTP news-server. By default, NewsPlex uses the standard port 119. |
| -s | Do not establish an NNTP news-server. |
| -w port | Create a Web server on port port for controlling NewsPlex from a Web browser. The port value of 80 is the standard value you can use. |
| -A | Read the config/authinfo file and set the login names and passwords as indicated in it. This avoids passing this information on the command line, where people can read it sometimes. |
| -P loginpassword | This option makes NewsPlex require entering the specified login name and password before the extended commands (NPsomething) are available. By default, all commands are available. This option also protects the Web access. |
| -U loginpassword | This option makes NewsPlex require entering the specified login name and password before any other commands can be entered. By default, the login and password are not required. This option also protects the Web access. |
| Option | Description |
|---|---|
| -n | Renumber articles if news-server unreachable. This is the equivalent of the NPRENUMBER command. Specifying this option once causes the renumbering level to be set to 1, specifying it twice causes the level to be set to 2, and specifying it 3 times sets the level to 3. |
| -E | Show only active news-groups to the news-readers. It is still possible to activate other news-groups. This option eg. allows to check whether your news-reader is subscribed to all active news-groups. |
| -F | When a news-group is activated, NewsPlex creates two activation announcement articles inside, of which the first one is visible to the news-reader. When the -F command-line option is used, the news-groups remain empty until some real articles are found for them on news-servers. |
| Option | Description | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| -i | Shutdown NewsPlex if the Internet connection is lost. This option works only if auto-updating is active (-u not specified). | ||||||||||
| -t |
Shorten file names to 8.3 conventions.
This options makes NewsPlex create files following the MS-DOS naming convention, so that NewsPlex can run on file-systems which do not support long filenames, for example FAT on OS/2 or s5 on Unix. It is not fully implemented yet, since in the dataset2 and index directories filenames still reflect the news-group names. |
||||||||||
| -v | Be more verbose and log additional information to the log files. This information notably includes performance data. | ||||||||||
| -z zombiedays | Set the deleted article disappearance delay to zombiedays days. Value of 0 disables zombie creation. The default zombiedays is 7. | ||||||||||
| -I |
Assume that the Internet is always present. Use this option
if NewsPlex is unable to correctly detect whether you are
connected to the Internet at a given moment, and notably
thinks that you are not while you really are.
Using this option will clutter the logs more easily with uninteresting host unreachable information, and also will cause the renumbering (see the NPRENUMBER command) to be performed when you try to request an article while not being connected to the Internet. |
||||||||||
| -R flags |
Configure the remembering of retrieved articles.
The flags are a series of letters, which
individually select the following options:
See the section below for more details. All flags are enabled by default. Note that in version 3.7 and before, the default processing was equivalent to s, that is, no logging was done, but already marked articles were shown as such. Setting the -R command-line option was equivalent to lms today. Warning: If you use Netscape or Mozilla to read news, then if the From: line of an article contains just an address without name, that is somebody@host, when the field is marked, the news-reader will display just the /r itself, probably because adding the prefix violates the format of Internet email addresses. |
| Option | Description |
|---|---|
| -l | Always request the full news-group list from news-servers when exploring them, and filter interesting news-groups locally. |
| -m |
Perform minimal exploration of news-servers and terminate.
This command causes NewsPlex to run in a special
Minimal mode, and only
rate the news-servers listed in the config/servers file.
NewsPlex will try to connect to each of them, and retrieve the latest available article's head in up to 20 news-groups from the list of news-groups matching the config/groups.yes file. If no article could be retrieved, the entire session will be logged into logs/unusable. Otherwise, the session log will stay in explore/server-disk-name. Note that NewsPlex will stop sampling the news-groups on a news-server if the connection with it has been broken temporarily. No dataset2, index or newsrc2 directories are used or created. |
| -u | Do not explore the news-servers automatically. It is still possible to explore a specific news-server by using the NPEXPLORESERVER command. |
| -x |
Do not use the XGTITLE NNTP protocol request
for getting the news-groups list.
NewsPlex normally does not use XGTITLE for getting the list of news-groups matching a pattern. However, if the news-server does not support retrieving names of news-groups using wild-cards, then XGTITLE will be used. Since this command is often broken and does not return all the possible news-groups, the -x option allows to avoid using it. NewsPlex will instead get the full news-groups list and filter locally. |
| -y days | Limit the age of article descriptions added to the article database during the news-server exploration to days days. By default, the age is not limited. This command-line option can be superseded for each news-server individually using the y per-news-server flag. See also the NPDELETEOLD command. |
| -D | If this option is specified, NewsPlex will look at the dates of the newsrc2 files and defer the exploration of news-servers for which the date of their file is recent. If you restart NewsPlex after a short interruption, it is better to specify this option. |
| -S | Skip news-groups which appear as empty in the global news-group list when exploring news-servers. By default, NewsPlex tries to explore news-groups which appear as empty by other means, in order to find all the articles they might contain in spite of possible incoherences in the information provided by the news-server, but this is not worth it most of the time. |
| -T count | Explore count news-servers at once instead of default. By default, up to 8 news-servers are explored at once in normal mode, and 16 in minimal mode. This is suitable for dial-up modem connections. Increasing this number can speed up things up if you have a fast Internet connection. Otherwise, it could make things worse because of Internet connection contention and resulting timeouts. Also, the more news-servers are explored at once, the more CPU and memory are used. |
| Option | Description |
|---|---|
| -b | Run in background.
By default, NewsPlex runs in foreground and blocks the window from which it has been started. The -b option makes NewsPlex put itself into the background. |
| -d dir | Use directory dir as work directory for files, rather than the directory from which NewsPlex is started. |
| -f | Delete unwanted articles. If this option is passed, the config/filters file is read on startup and all articles which match the patterns inside are deleted from the article database. |
| -h | NewsPlex will print a short help about command-line options and exit. |
|
-ol -oi |
Lower the priority of the NewsPlex process to the Idle level; both options are equivalent. This frees the CPU, but the download speeds might suffer. NewsPlex could become CPU-starved if you already have programs running in the background permanently and at a low priority, like Seti@Home, or if you run DOS programs which take all CPU. On Windows NT/2000/XP, you can also adjust the priority of NewsPlex from the Task Manager. |
| -r |
The list of news-servers is raw.
If this option is specified, NewsPlex expects a simpler format for the config/servers list of news-servers, without unique server-numbers in first column:
server-url [flags] [# comment]
Entries with the same server-url are merged:
each news-server is explored only once even if present multiple times
in the list.
It is possible to use the logs/results
file as a raw config/servers file for the
next exploration.
You should not use the raw format during normal operation because the numbering of news-servers will change if you delete a news-server, while in fact the news-server numbers are used in the index and update files. |
| -M | This option makes NewsPlex read the etc/retrievd file on startup and mark as retrieved (see section below) all the article descriptions whose Message-IDs are listed in this file. This can also be performed using the NPAPPLYRETRIEVD command at any time. |
| -N | Do not wait for a final key press before exiting. |
| -O | Compact all the databases and rewrite all index files at startup (merging information from update files in). This option allows to bring the databases to the smallest size. It however takes a certain time to execute. |
If passed the -e command-line option, NewsPlex will allow NNTP and Web logins from any machine. This will completely remove any access protection.
It is possible to protect both NNTP and Web access with passwords. The passwords are the same for both accesses.
Two login and password couples can be used to separately protect user and supervisor (administrative) access.
Web access is protected with the supervisor login and password if a supervisor login and password are set, and with the user login and password if the user login and password are set and the supervisor login and password are not set. When the correct login and password are provided, all functions become available.
If the -P login password command-line option is passed, supervisor NNTP access will be protected and only standard commands will be available to users which did not provide the correct values. People will notably not be able to see what news-servers are used by NewsPlex. If the -U login password command-line option is passed, even normal user access will be protected and only the authentication command will be available until correct authentication data are provided. If -U is passed without an additional -P, every user will be privileged as soon as it logs in. By passing both -U and -P, one can separately protect both normal access and supervisor access. If a client news-reader does not have supervisor access, retrieved article headers are stripped of Path and Xref entries, which give the identity of the news-servers from which the articles come. The Path entry is useless normally and the Xref entry mostly too, since it contains article numbers on the originating news-server rather than article numbers on NewsPlex. Note that in the asynchronous article retrieval case, the stripping is done on the fly, depending on the privilege of the news-reader, and the async/xxxxxxxx.ok files on disk are complete.
To delete a news-server, for example because it has been unusable for a certain time, first delete the article descriptions which still point to it, using the NPDELETESERVER command, then delete or comment out its entry in the config/servers file and restart NewsPlex.
This feature is enabled by default starting from version 3.8. It can be controlled with the -R flags command-line option. It works as follows:
Each time you retrieve an article successfully, NewsPlex stores its Message-ID into the etc/retrievd file, and marks the article in the article database.
If a news-reader later requests a full list of articles for a news-group, it will get modified article descriptions: NewsPlex will add a "/r " prefix to the From field of the article descriptions. If an article was previously from John Doe <john@doe.com> it will become an article from /r John Doe <john@doe.com>.
The articles themselves are not modified when they are actually retrieved.
Normally, news-readers only request new article descriptions, so they do not see the modified article descriptions unless they request everything.
The etc/retrievd file allows to keep the information even after the article has vanished from all news-servers where it was present (and the zombie has expired too, or zombies were not enabled), and hence the article description, containing the retrieved status has been deleted.
The -M command-line option makes NewsPlex read the etc/retrievd file on startup or restart and mark as retrieved all the articles whose Message-IDs are listed in this file.
This command-line option is useful if you add a new news-server to the config/servers file and it has articles which you have already retrieved in the past, but which do not exist in the article database anymore. Except for the date, such articles would appear as new. If the -M command-line option is specified, NewsPlex will mark these old articles with "/r " and you will be able to avoid retrieving them again.
There is presently no way of doing the opposite, that is transferring the Message-IDs of previously retrieved articles from the article database into a file.
In order to migrate to version 3.5 manually, you could perform the following steps:
| File | Purpose |
|---|---|
| config/groups.yes | If you want to limit the memory usage and are not interested in the full group list |
| config/servers | If you have additional news-servers |
| config/filters | If you defined some filters previously |
| etc/retrievd | If you want to retain the information about which article have been read |
Do not keep the config/groups.no or the config/groups.fix files, as they as no more necessary. config/groups.no now just hides groups and consumes additional memory for groups which might not even exist anymore. config/groups.fix is no more used at all.
| Code | Meaning |
|---|---|
| 0 | "No problem" |
| 1 | "Help was printed" |
| 2 | "Bad option was passed" |
| 3 | "Missing work directory" |
| 4 | "Could not open the journal file" |
| 5 | "Another server already running on same port" |
| 6 | "The config directory does not exist" |
| 7 | "Option -A passed but no authinfo file" |
| 8 | "Incorrect authinfo file" |
| 9 | "Missing groups.yes file" |
| 10 | "Could not create the logs directory" |
| 11 | "Could not create the etc directory" |
| 12 | "Could not create the list directory" |
| 13 | "Could not create the explore directory" |
| 14 | "Could not create the async directory" |
| 15 | "Option -t not specified but disk is 8.3 only" |
| 16 | "Could not create the newsrc2 directory" |
| 17 | "Could not create the index directory" |
| 18 | "Could not create the dataset directory" |
| 19 | "Could not create the unusable file" |
| 20 | "Error in the servers file" |
| 21 | "Error when processing the groups.* files" |
| 22 | "Error when initializing asynchronous retrieval" |
| 23 | "Could not establish an NNTP server" |
| 24 | "Could not establish a Web server" |
| 25 | "Could not create async retrieval threads" |
| 26 | "Error when converting the databases" |
| 27 | "Shutdown requested during startup" |
| 28 | "Could not create the NNTP server thread" |
| 29 | "Could not create the Web server thread" |
| 30 | "Could not create the compaction task" |
| 32 | "Could not create the memory monitor task" |
| 33 | "Article noticed as corrupted during article free operation" |
| 34 | "Article noticed as corrupted during article save operation" |
| 35 | "Error during synchronization with main server" |
| 36 | "Internet not present" |
| 37 | "An async retrieval task could not be created" |
| 38 | "The log file for NNTP connection could not be created" |
| 39 | "The NPSHUT command has been performed" |
| 40 | "Cannot initialize the logs/clients file" |
| 41 | "Cannot initialize the logs/web file" |
| 42 | "Shutdown requested from the web interface" |
| 43 | "Shutdown requested by signal" |
| 45 | "Could not create the update directory" |
| 46 | "Article noticed as corrupted during Message-ID compare" |
| 47 | "Auto-configuration failed" |
| 48 | "Could not create the server update task" |
| 49 | "Automatic shutdown after exploring all servers in minimal mode" |
| 50 | "Could not read the etc/popular file" |
| 51 | "Could not repair the article database" |
| 52 | "NNTP server and Web server ports not different" |
I recompiled the kernel uping some hard coded limits on threads and file descriptors and there are no more (Out of resources). You might want to note following in the Docs: /usr/include/linux/limits.h NR_OPEN -> 4096 /usr/include/linux/posix_types.h __FD_SETSIZE -> 4096 in /etc/security.conf add *soft nofile 2048 *hard nofile 4096 in /etc/pam.d/logon add session required /lib/security/pam_limits.so in /usr/include/linux/tasks.h NR_TASKS = 4090 MIN_TASKS_LEFT_FOR_ROOT = 90 Note MAX_TASKS_PER_USER is limited by glibc to 1000... The above is for debian frozen potato at 2.2.17pre3Another hint from Ed:
I got crashes with kernels below 2.2.17pre3. If your system crashes shortly after starting NewsPlex (it could take up to half an hour here) try upgrading your kernel.
Stable new Windows versions were in the past uploaded to the ftp://ftp.cdrom.com/pub/simtelnet/news site (Simtel.Net), and sometimes to the http://www.winsite.com site (CICA). In the first case, an announcement was automatically posted to comp.archives.ms-windows.announce.
Stable new OS/2 versions were occasionally uploaded to the ftp://ftp-os2.nmsu.edu/pub/os2/apps/internet/news/server site.
The Solaris and FreeBSD versions can only be obtained from me directly, by email.
NewsPlex is sometimes mentioned or discussed in the news-groups ( alt.usenet.offline-reader.forte-agent, news.software.readers, it.comp.software.newsreader and others). You can use www.deja.com and enter newsplex as search keyword to locate them.
Joseph Morlan has recently started a Yahoo Group (mailing list with Web-access) dedicated to NewsPlex at http://groups.yahoo.com/group/NewsPlex/. As of this writing, it has over 110 members.
Some people maintain references to NewsPlex on their pages. There is also an Italian translation of the manual: