New HYPTUSS versions 11.86 (Excel) and c-b (LibreOffice)

[Fluke]

Just double checked my existing version (c-a), that is working fine on my set up here. So something seems to have gone awry between c-a and c-b, for me at least.

[kiloran]

I'm not sure what's happening here.
The video was created using Excel on Windows, so there are some minor differences for LibreOffice on other OS.
I'm using LibreOffice on Windows 11 and when I pasted the new data sheet using Command+V as you did, it pasted all of the data in Column A of the data sheet, which is clearly wrong. On LibreOffice, use right-click / Paste Special / Unformatted text, select TAB as the separator and click OK .
An easier way is to use the more automated process. Download the data sheet to your PC (typically the Downloads folder, but anywhere will do). Then select cell A1 and click on the box in cell A1 "Select your start row etc". This provides a file selection window. Locate your downloaded file and click on OK and the data will be updated. See page 17 of the User Guide.

However, this does not explain your problem. Updating share prices does not actually use the data sheet. You can delete all the data in the data sheet and you will still get updated prices when you click on the "Get Prices" button (IF you are using Yahoo as the price source, which is the default)

It sounds like you have corrupted something, so I suggest downloading a fresh copy of HYPTUSS and try again.

--kiloran

Hi Kiloran, yes take your point about the get price having nothing to do with the data sheet, think I meant to add a new share but anyway. I've done as you suggested, deleted the one from yesterday and just downloaded a new copy, Ignoring the data sheet problem for now I went straight to the HYP sheet and tried to get the price for the single BP share and got the following error message. I didn't touch anything on the parameters sheet.

com.sun.star.uno.RuntimeException: Error during invoking function getPriceSrc in module vnd.sun.star.tdoc:/13/Scripts/python/HypTopUp.py (<class 'UnboundLocalError'>: local variable 'response' referenced before assignment
File "/Applications/LibreOffice.app/Contents/Resources/pythonscript.py", line 915, in invoke
ret = self.func( *args )
File "vnd.sun.star.tdoc:/13/Scripts/python/HypTopUp.py", line 4811, in getPriceSrc
File "vnd.sun.star.tdoc:/13/Scripts/python/HypTopUp.py", line 2060, in HypUpdYahooPriPy_3
)

That does sound odd. I've downloaded a fresh copy of version c-b and it works fine on Windows 10, Windows 11 and Linux Mint. Don't have a Mac so can't check that.
From your feedback, it sounds like your copy is having problems retrieving data from the Yahoo web page, though I've no idea why since it uses the same subroutine to retrieve data from all websites, and that subroutine has not changed from version c-a and many earlier versions.

Download a fresh copy of c-b, then go to the parameters sheet and unhide row 9, Change cell B9 from 0 to 1. This will enable an unpublicised debugging mode. Then go to the High Yield Portfolio sheet and click on the Get Prices button. This will write some debugging data to a file HYPTUSS_log.txt in the same folder as your HYPTUSS. Open that file and you should get something like this:

Code: Select all

15:44:30 Line: 4803 ===================================================
 
15:44:30 Line: 4806 Starting priceSrc: Yahoo
15:44:30 Line: 2020 
15:44:30 Line: 2020 >>>>>>>>> Starting HypUpdYahooPriPy_3
15:44:30 Line: 2020 Python sys version: 3.8.1
15:44:30 Line: 2046 HypUpdYahooPri_3 url = https://uk.finance.yahoo.com/quotes/BP.L,USDGBP=X,EURGBP=X/view/v1
15:44:32 Line: 2068 PriceDict = {'BP.L': ['508.6', 'GBp'], 'USDGBP=X': ['0.78907', 'GBP'], 'EURGBP=X': ['0.85783', 'GBP']}

Send that data to me.
A warning.... anything further you do in HYPTUSS will append more debugging data to that file, so before you do anything else, set cell B9 on the Parameters sheet back to 0

Now, on the Parameters sheet, set cell B4 to SharePrices. This will retrieve prices from the SharePrice website with exactly the same functionality as version c-a. Does that still work for you on version c-b?

I wonder if you have an antivirus tool or whatever which is perhaps blocking the Yahoo site. What happens if you click on this link:
https://uk.finance.yahoo.com/quotes/BP. ... =X/view/v1
Do you get the Yahoo web page showing data for BP and the two exchange rates?

That's all I can think of right now. Your problem does seem odd.

--kiloran

[csearle]

Any idea why the Latest Forecast yield is 570% for BHP? It is the only outlier. The data is being scraped correctly, just their website insists this is the forecast. They can clearly see something I cant! C.

[Fluke]

I wonder if you have an antivirus tool or whatever which is perhaps blocking the Yahoo site. What happens if you click on this link:
https://uk.finance.yahoo.com/quotes/BP. ... =X/view/v1
Do you get the Yahoo web page showing data for BP and the two exchange rates?

That's all I can think of right now. Your problem does seem odd.

--kiloran

The link works fine kiloran, I'll follow your instructions above and get back to you.

[kiloran]

Any idea why the Latest Forecast yield is 570% for BHP? It is the only outlier. The data is being scraped correctly, just their website insists this is the forecast. They can clearly see something I cant! C.

Perhaps they know the share price is going to plummet shortly. Or there is going to be a truly massive increase in the dividend (fill your boots NOW!)

More seriously, Sharecast do have a habit of getting some data massively wrong, seemingly more than most other websites. BHP isn't the only share with such a grotesque forecast yield, there's another 5-6 of them in the FTSE350. Over the years, I've reported this kind of problem to them on several occasions, and never had a response or fix.

At least we're not paying for the data.

--kiloran

[Arborbridge]

Any idea why the Latest Forecast yield is 570% for BHP? It is the only outlier. The data is being scraped correctly, just their website insists this is the forecast. They can clearly see something I cant! C.

I've taken to the habit of just dividing this by 100 - lazy, I know and I should check this properly. If it were critical for some decision, I would, meanwhile it is closer than 570%!


Arb

[Fluke]

That does sound odd. I've downloaded a fresh copy of version c-b and it works fine on Windows 10, Windows 11 and Linux Mint. Don't have a Mac so can't check that.
From your feedback, it sounds like your copy is having problems retrieving data from the Yahoo web page, though I've no idea why since it uses the same subroutine to retrieve data from all websites, and that subroutine has not changed from version c-a and many earlier versions.

Download a fresh copy of c-b, then go to the parameters sheet and unhide row 9, Change cell B9 from 0 to 1. This will enable an unpublicised debugging mode. Then go to the High Yield Portfolio sheet and click on the Get Prices button. This will write some debugging data to a file HYPTUSS_log.txt in the same folder as your HYPTUSS. Open that file and you should get something like this:

Code: Select all

15:44:30 Line: 4803 ===================================================
 
15:44:30 Line: 4806 Starting priceSrc: Yahoo
15:44:30 Line: 2020 
15:44:30 Line: 2020 >>>>>>>>> Starting HypUpdYahooPriPy_3
15:44:30 Line: 2020 Python sys version: 3.8.1
15:44:30 Line: 2046 HypUpdYahooPri_3 url = https://uk.finance.yahoo.com/quotes/BP.L,USDGBP=X,EURGBP=X/view/v1
15:44:32 Line: 2068 PriceDict = {'BP.L': ['508.6', 'GBp'], 'USDGBP=X': ['0.78907', 'GBP'], 'EURGBP=X': ['0.85783', 'GBP']}

Send that data to me.
A warning.... anything further you do in HYPTUSS will append more debugging data to that file, so before you do anything else, set cell B9 on the Parameters sheet back to 0

Now, on the Parameters sheet, set cell B4 to SharePrices. This will retrieve prices from the SharePrice website with exactly the same functionality as version c-a. Does that still work for you on version c-b?

I wonder if you have an antivirus tool or whatever which is perhaps blocking the Yahoo site. What happens if you click on this link:
https://uk.finance.yahoo.com/quotes/BP. ... =X/view/v1
Do you get the Yahoo web page showing data for BP and the two exchange rates?

That's all I can think of right now. Your problem does seem odd.

--kiloran

Here you go kiloran, your example has an additional line.


12:40:05 Line: 4803 ===================================================

12:40:05 Line: 4806 Starting priceSrc: Yahoo
12:40:05 Line: 2020
12:40:05 Line: 2020 >>>>>>>>> Starting HypUpdYahooPriPy_3
12:40:05 Line: 2020 Python sys version: 3.8.1
12:40:05 Line: 2046 HypUpdYahooPri_3 url = https://uk.finance.yahoo.com/quotes/BP. ... =X/view/v1

[Fluke]

Now, on the Parameters sheet, set cell B4 to SharePrices. This will retrieve prices from the SharePrice website with exactly the same functionality as version c-a. Does that still work for you on version c-b?

--kiloran

Yep. Works fine when set to SharePrice

[kiloran]

Here you go kiloran, your example has an additional line.

12:40:05 Line: 4803 ===================================================

12:40:05 Line: 4806 Starting priceSrc: Yahoo
12:40:05 Line: 2020
12:40:05 Line: 2020 >>>>>>>>> Starting HypUpdYahooPriPy_3
12:40:05 Line: 2020 Python sys version: 3.8.1
12:40:05 Line: 2046 HypUpdYahooPri_3 url = https://uk.finance.yahoo.com/quotes/BP. ... =X/view/v1

Thanks Fluke. So, we now know that HYPTUSS is creating the correct URL, but for some odd reason is not getting data back from that URL, even though that URL is fine in your browser.
I'm struggling to understand why that should be, when the data retrieval function for a URL is the same for all web queries including SharePrice, Sharecast and whatever.
It's addled my brain somewhat, so I'll go and sit in a dark corner to recover, then have a another think.

--kiloran

[Fluke]

Here you go kiloran, your example has an additional line.

12:40:05 Line: 4803 ===================================================

12:40:05 Line: 4806 Starting priceSrc: Yahoo
12:40:05 Line: 2020
12:40:05 Line: 2020 >>>>>>>>> Starting HypUpdYahooPriPy_3
12:40:05 Line: 2020 Python sys version: 3.8.1
12:40:05 Line: 2046 HypUpdYahooPri_3 url = https://uk.finance.yahoo.com/quotes/BP. ... =X/view/v1

Thanks Fluke. So, we now know that HYPTUSS is creating the correct URL, but for some odd reason is not getting data back from that URL, even though that URL is fine in your browser.
I'm struggling to understand why that should be, when the data retrieval function for a URL is the same for all web queries including SharePrice, Sharecast and whatever.
It's addled my brain somewhat, so I'll go and sit in a dark corner to recover, then have a another think.

--kiloran

Well it's Friday evening so perhaps a glass of wine would help? Something funny's going on but I've got a couple of workarounds, carry on with the c-a version or use c-b with SharePrices. So not a major problem. Nobody else seems that bothered.

[csearle]

I'm struggling to understand why that should be, when the data retrieval function for a URL is the same for all web queries including SharePrice, Sharecast and whatever.
It's addled my brain somewhat, so I'll go and sit in a dark corner to recover, then have a another think.

Just a thought: do browsers, as part of their connexion with a server, supply cookies when requested? If they do could it be that the browser responds with the cookie but the non-browser request doesn't? C.

[torata]

Is this a bug or just me?

I've been using LO c-b since Sept 23.
I noticed from quite early on that the many of the Dividend tab updates seemed to be very much out of date - often a year or 6 months - using the Sharecast feed. Probably for most of the shares on my portfolio.
I always assumed it was just Sharecast, but in fact today going to the Sharecast webpage from the button on the Portfolio tab for some of the offending shares, shows that the Sharecast page is up-to-date.

Egs from my Dividend tab:
AstraZeneca shows as 11 Sep 23, which is the previous dividend, not 25 Mar 24
Tritax Bigbox shows as 01 Jun 23, which is 5 dividends previous, not 02 Apr 24 (which is latest shown on Sharecast)
However Titax Eurobox shows correctly at 14 Mar 24

So for some reason even though HYPTUSS looks like it's updating, it seems to be not actually updating for most shares, but maybe pulling some of the data from a cache somewhere?

Any idea what this could be?

Re using the other sources: I did try changing to ADVFN, but got a few gaps, so gave up. Dividend data brings in about the first 12 or so shares, then returns a null, so I reluctantly gave up on that also.

torata

[torata]

Is this a bug or just me?

So for some reason even though HYPTUSS looks like it's updating, it seems to be not actually updating for most shares, but maybe pulling some of the data from a cache somewhere?

Any idea what this could be?

torata

FWIW, I did run the debug script as outlined above, if it helps.
torata

[niord]

Is this a bug or just me?

I've been using LO c-b since Sept 23.
I noticed from quite early on that the many of the Dividend tab updates seemed to be very much out of date - often a year or 6 months - using the Sharecast feed. Probably for most of the shares on my portfolio.
I always assumed it was just Sharecast, but in fact today going to the Sharecast webpage from the button on the Portfolio tab for some of the offending shares, shows that the Sharecast page is up-to-date.

Egs from my Dividend tab:
AstraZeneca shows as 11 Sep 23, which is the previous dividend, not 25 Mar 24
Tritax Bigbox shows as 01 Jun 23, which is 5 dividends previous, not 02 Apr 24 (which is latest shown on Sharecast)
However Titax Eurobox shows correctly at 14 Mar 24

So for some reason even though HYPTUSS looks like it's updating, it seems to be not actually updating for most shares, but maybe pulling some of the data from a cache somewhere?

Any idea what this could be?

Re using the other sources: I did try changing to ADVFN, but got a few gaps, so gave up. Dividend data brings in about the first 12 or so shares, then returns a null, so I reluctantly gave up on that also.

torata

I actually get the same on those sources but I am happy waiting for Dividend data as I mainly have investment trusts and less than 12 so the null response has never been an issue.

[kiloran]

Is this a bug or just me?

I've been using LO c-b since Sept 23.
I noticed from quite early on that the many of the Dividend tab updates seemed to be very much out of date - often a year or 6 months - using the Sharecast feed. Probably for most of the shares on my portfolio.
I always assumed it was just Sharecast, but in fact today going to the Sharecast webpage from the button on the Portfolio tab for some of the offending shares, shows that the Sharecast page is up-to-date.

Egs from my Dividend tab:
AstraZeneca shows as 11 Sep 23, which is the previous dividend, not 25 Mar 24
Tritax Bigbox shows as 01 Jun 23, which is 5 dividends previous, not 02 Apr 24 (which is latest shown on Sharecast)
However Titax Eurobox shows correctly at 14 Mar 24

So for some reason even though HYPTUSS looks like it's updating, it seems to be not actually updating for most shares, but maybe pulling some of the data from a cache somewhere?

Any idea what this could be?

Re using the other sources: I did try changing to ADVFN, but got a few gaps, so gave up. Dividend data brings in about the first 12 or so shares, then returns a null, so I reluctantly gave up on that also.

torata

HYPTUSS looks for the first record in the dividend data table on Sharecast. I am sure that when the Sharecast dividend functionality was added to HYPTUSS about 4 years ago, the first record was the most recent payment. Now, Sharecast displays dividends in date sequence for the current fiscal year, so if there have been two payments, the most recent is the second record. If there have been three payments, the most recent is the third record, etc. If there have been more than one payment, HYPTUSS will not pick of the most recent.

It would be possible to rewrite the code to find the most recent payment, but this is a bit messy, and in the light of the announcement that support for HYPTUSS is now withdrawn, I'll sadly have to reject a fix for this bug. ADVFN is the best alternative for dividends. DividendData does provide the best quality data, but it dislikes web-scraping and will lock you out if you have more than about 12 shares.

Sorry!

--kiloran